企业官网建设流程全解析

Source Code Build

【免费下载链接】hcommHCOMM（Huawei Communication）是HCCL的通信基础库，提供通信域以及通信资源的管理能力。项目地址: https://gitcode.com/cann/hcomm

Environment Preparation

This project supports source code building. Before compiling and running, complete the basic environment setup and source code download by following the steps below. Ensure that the NPU driver, firmware, and CANN software are installed.

Prerequisites

The software dependencies required for compiling this project are listed below. Ensure that the version requirements are met.

• python >= 3.7.0
• pip3 >= 20.3.0
• setuptools >= 45.0.0
• wheel >= 0.34.0
• gcc and g++: 7.3.0 to 13.3.x
• cmake >= 3.16.0
• pkg-config >= 0.29.1 (for compiling rdma-core)
• patch >= 2.7.0 (for applying patch files)
• ccache (optional, for improving incremental compilation speed)
• lcov (optional, for generating UT or ST coverage reports)

Installing the CANN Software Package

1. Install the driver and firmware (runtime dependency)

   For downloading and installing the driver and firmware, refer to the "Prepare Software Package" and "Install NPU Driver and Firmware" sections in the CANN Software Installation Guide. The driver and firmware are runtime dependencies. If you are only compiling the source code of this project, they do not need to be installed.

2. Install the CANN software package

   • Scenario 1: Experience or develop based on the master version

     Click the download link, select the latest version, and download the corresponding package based on the product model and environment architecture. The installation commands are as follows. For more guidance, refer to the CANN Software Installation Guide.

     1. Install the CANN Toolkit development kit package.

        # Ensure the installation package has executable permissions chmod +x Ascend-cann-toolkit_${cann_version}_linux-${arch}.run # Installation command ./Ascend-cann-toolkit_${cann_version}_linux-${arch}.run --install --install-path=${install_path}

     2. Install the CANN ops operator package (runtime dependency).

        The ops operator package is a runtime dependency. If you are only compiling the source code of this project, this package does not need to be installed.

        # Ensure the installation package has executable permissions chmod +x Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run # Installation command ./Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run --install --install-path=${install_path}

     • ${cann_version}: Indicates the CANN software package version number.
     • ${arch}$: Indicates the CPU architecture, for example, aarch64 or x86_64.
     • ${soc_name}: Indicates the NPU model name.
     • ${install_path}: Indicates the specified installation path. The CANN ops operator package must be installed in the same path as the CANN Toolkit development kit package. The default installation path for the root user is/usr/local/Ascend.

   • Scenario 2: Experience or develop based on a released version

     Visit the CANN official download center, select a released version (only CANN 8.5.0 and later versions are supported), and download the corresponding package based on the product model and environment architecture. Finally, follow the commands provided on the webpage to complete the installation.

Environment Verification

After installing the CANN software package, verify that the environment is functioning correctly.

• Check the NPU device:

  # Run npu-smi. If device information is displayed normally, the driver is working correctly. npu-smi info

• Check the CANN software:

  # View the version information provided by the version field of the CANN Toolkit development kit package (default installation path). <arch> indicates the CPU architecture (aarch64 or x86_64). cat /usr/local/Ascend/cann/<arch>-linux/ascend_toolkit_install.info # View the version information provided by the version field of the CANN ops operator package (default installation path). cat /usr/local/Ascend/cann/<arch>-linux/ascend_ops_install.info

Environment Variable Configuration

Select the appropriate command to apply the environment variables.

# Default installation path, using the root user as an example (for non-root users, replace /usr/local with ${HOME}) source /usr/local/Ascend/cann/set_env.sh # Specified installation path # source ${install_path}/cann/set_env.sh

Compilation and Installation

Downloading the Source Code

Use the following command to download the source code. Replace ${tag_version} with the target branch tag name. For the mapping between source branch tags and CANN versions, refer to the release repository.

# Download the source code for the corresponding project branch git clone -b ${tag_version} https://gitcode.com/cann/hcomm.git

Compiling the Source Code

This project provides a one-click build capability. Navigate to the root directory of the repository and execute the following commands:

# Compile the host package bash build.sh --pkg # Compile the host and device package bash build.sh --pkg --full

During compilation, the dependency packages listed in Open Source Third-Party Software Dependencies are automatically downloaded. If the compilation environment does not have network access, download the required dependency packages in a networked environment, manually upload them to the compilation environment, and specify the dependency package storage path using the--cann_3rd_lib_pathparameter.

# Specify the dependency package storage path. Default: ./third_party bash build.sh --cann_3rd_lib_path={your_3rd_party_path}

After compilation, acann-hcomm_<version>_linux-<arch>.runsoftware package is generated in the./build_outdirectory.

<version>indicates the software version number, and<arch>indicates the operating system architecture. The values include x86_64 and aarch64.

Installation

Install the compiled HCOMM software package:

bash ./build_out/cann-hcomm_<version>_linux-<arch>.run --full

During installation, replace the package name in the command with the actual software package name.

After installation, the HCOMM software package generated by the user replaces the HCOMM-related software in the installed CANN Toolkit development kit package.

Uninstallation

To uninstall the compiled HCOMM software package and restore it to the state after installing the CANN Toolkit development kit package, use the following command:

bash ./build_out/cann-hcomm_<version>_linux-<arch>.run --uninstall

During uninstallation, replace the package name in the command with the actual software package name.

Testing

LLT Testing

After installing the compiled HCOMM software package, execute the following command to run LLT test cases:

bash build.sh --ut

On-Board Testing

NoteBefore on-board testing, ensure that the driver, firmware, CANN Toolkit development kit package, and CANN ops operator package are installed.

Developers can use the HCCL Test tool for collective communication function and performance testing on the board. The workflow for using the HCCL Test tool is as follows:

1. Tool compilation

   Before using the HCCL Test tool, install the MPI dependency and compile the HCCL Test tool. For detailed instructions, refer to the "MPI Installation and Configuration" and "Tool Compilation" sections in the corresponding version of the Ascend Documentation Center - HCCL Performance Test Tool Guide.

2. Disable signature verification

   Thecann-hcomm_<version>_linux-<arch>.runsoftware package generated from this source repository contains the following tar.gz subpackages:

   • cann-hcomm-compat.tar.gz: HCOMM compatibility upgrade package.
   • cann-hccd-compat.tar.gz: DataFlow compatibility upgrade package.
   • aicpu_hcomm.tar.gz: AI CPU communication base package.

   These tar.gz packages are loaded to the Device when the service starts. During the loading process, the driver performs security signature verification by default to ensure the package is trusted. Because the tar.gz packages compiled from this source repository do not contain a signature header, the driver security signature verification mechanism needs to be disabled.

   Method to disable signature verification:

   Use Ascend HDK 25.5.T2.B001 or later, and use the npu-smi tool provided with the Ascend HDK to disable signature verification. The following are reference commands. Execute them as the root user on the physical machine (using device 0 as an example).

   npu-smi set -t custom-op-secverify-enable -i 0 -d 1 # Enable signature verification configuration npu-smi set -t custom-op-secverify-mode -i 0 -d 0 # Disable custom signature verification

3. Execute the HCCL Test command to test the function and performance of collective communication.

   Using one compute node, 8 NPU devices, and testing the AllReduce operator performance as an example:

   # /usr/local/Ascend is the CANN software installation path for the root user under the default installation path. Replace it with the actual path. cd /usr/local/Ascend/ascend-toolkit/latest/tools/hccl_test # Data size (-b) from 8KB to 64MB, increment factor (-f) of 2, number of NPUs participating in training is 8 mpirun -n 8 ./bin/all_reduce_test -b 8K -e 64M -f 2 -d fp32 -o sum -p 8

   For detailed usage instructions of the tool, refer to the "Tool Execution" section in the Ascend Documentation Center - HCCL Performance Test Tool Guide.

4. View the results

   After executing the HCCL Test tool, the display output is as follows:

   • "check_result" shows success, indicating that the communication operator executed successfully and the AllReduce operator function is correct.
   • "aveg_time": The execution time of the collective communication operator, in us.
   • "alg_bandwidth": The execution bandwidth of the collective communication operator, in GB/s.
   • "data_size": The amount of data participating in collective communication on a single NPU, in Bytes.

Appendix

Open Source Third-Party Software Dependencies

When compiling this project, the following third-party open source software dependencies are required:

【免费下载链接】hcommHCOMM（Huawei Communication）是HCCL的通信基础库，提供通信域以及通信资源的管理能力。项目地址: https://gitcode.com/cann/hcomm