Skip to content

Commit 37823b4

Browse files
aayush-fadianevalsar
aayush-fadia
and
nevalsar
authored
Add Building Software from Source tutorial (#221)
- Create new tutorial on building libraries from source (OpenCV example) - Update navigation and programming category index - Ensure compliance with project standards (links, front matter) --------- Co-authored-by: Nevin Valsaraj <nevin.valsaraj32@gmail.com>
1 parent 0e0052e commit 37823b4

3 files changed

Lines changed: 224 additions & 0 deletions

File tree

‎_data/navigation.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,8 @@ wiki:
239239
url: /wiki/programming/tutorials-resources/
240240
- title: Python Construct
241241
url: /wiki/programming/python-construct/
242+
- title: Building Software from Source
243+
url: /wiki/programming/build-from-source/
242244
- title: ROS 2 Yasmin State Machine
243245
url: /wiki/programming/yasmin-ros2-state-machine/
244246
- title: Networking

‎wiki/programming/build-from-source.md‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
---
2+
date: 2025-05-04
3+
title: Building software from source
4+
---
5+
6+
This tutorial describes how to configure and build libraries from source for your unique system.
7+
This is often necessary when your software requirements are not met by the packages available in the repositories of
8+
your distribution (the version of the package you want is not available from `apt` for your version of Ubuntu, for
9+
example).
10+
This tutorial will cover the process for customizing and building packages from source as you need them.
11+
Along the way, common pitfalls and how to avoid them will be discussed.
12+
You will also discover how to avoid polluting your system environment to avoid conflicts between the packages you build
13+
and the ones installed by your package manager.
14+
15+
## Why build your packages from source?
16+
17+
There are several reasons you might want to build packages from source, a few of these are listed below:
18+
19+
- The package (+ version) you want is not available from your system package manager.
20+
- You need a customized build of the package (building OpenCV with CUDA support for example).
21+
- For optimized builds that fully utilize all the features your hardware has to offer.
22+
23+
## Process outline and key pieces
24+
25+
To build software from source code, these are the broad steps that must be followed:
26+
27+
- Obtain the source code: Usually `git clone` and `git checkout` and then apply patches if necessary.
28+
- Get dependencies: Use your system package manager or build the packages that this package depends on so they are
29+
available to this package.
30+
- Configure the build: Use `cmake` to set options for the build, including which features to compile with and the path
31+
where the package must be installed.
32+
- Build and install the package: Use `cmake` to build and install the package.
33+
- Linking against the built package: Set environment/cmake variables so that other packages can find the package
34+
that you just built.
35+
36+
This tutorial will use the popular computer-vision library OpenCV as a guiding example due to its popularity and
37+
complexity, which will concretely demonstrate all steps.
38+
39+
## Step 0: Prerequisites
40+
41+
To build software, the system needs to have the following installed:
42+
43+
- [CMake](https://cmake.org/)
44+
- [Ninja](https://ninja-build.org/) as a preferred build tool.
45+
- A C/C++ compiler, linker, and standard libraries/headers (usually preinstalled).
46+
47+
This collection is called the "toolchain".
48+
On Ubuntu all of this can be installed with:
49+
50+
```bash
51+
sudo apt install build-essential
52+
```
53+
54+
## Step 1: Getting the source code
55+
56+
### Which version do I require
57+
58+
If this package is being compiled to satisfy the dependency of other software that you need, you should check which
59+
exact version that dependent package needs. This can be done by analyzing the CMakeLists.txt file of the dependent (
60+
final) package, and finding a line resembling:
61+
62+
```cmake
63+
find_package(OpenCV 4 ...)
64+
```
65+
66+
This tells you that this package requires OpenCV 4 (Any 4.x.y version works).
67+
68+
Get source code for this version.
69+
70+
### Cloning source code
71+
72+
Get the source code on your computer into a well-organized folder (in a separate directory, that directory is alongside
73+
directories that contain source-code for other packages).
74+
75+
Instructions should be present on the package website. Usually, this involves:
76+
77+
- `git clone` the package.
78+
- `git checkout` to the release that is desired, e.g. `git checkout 4.11.0` to get that version of OpenCV, obtained from
79+
the GitHub releases page.
80+
81+
Here you would modify the source code (such as apply any quick fixes/patches as required).
82+
83+
**Also follow any prescribed instructions as the package demands**
84+
85+
To build OpenCV with the contrib feature set, we must also download and extract some more content. Follow instructions
86+
listed on the website.
87+
88+
## Step 2: Get dependencies
89+
90+
This library will depend on other software. Get that software, usually from your package manager, by running a command
91+
similar to (on ubuntu):
92+
93+
```bash
94+
sudo apt install lib-<libname>-dev
95+
```
96+
97+
For example, to get the "png" dependency to allow OpenCV to read png images, just run:
98+
99+
```bash
100+
sudo apt install libpng-dev
101+
```
102+
103+
You can proceed to the next step, and install these as CMake will complain that it did not find a dependency.
104+
105+
## Step 3: Configure the build
106+
107+
This is the crucial step. Here, you decide how the package needs to be compiled, this includes:
108+
109+
- Which features you need: In OpenCV, for example, which image formats (png, jpg, etc.) you want to support
110+
reading/writing to?
111+
- Do you want OpenCV to support CUDA?
112+
- Do you want to build support for Python?
113+
- Disable/enable features (GUI support, for example).
114+
115+
### Setting the environment
116+
117+
Be in the environment under which you will run this package.
118+
119+
Concretely, this means: If you want OpenCV to be built in a particular Python virtual environment, in your terminal,
120+
source this Python Environment.
121+
122+
Also set `CMAKE_PREFIX_PATH` (explained later) to allow CMake to find other packages.
123+
124+
### Set Options
125+
126+
Now, create a well-organized build directory, where build time intermediates will be stored (these are temporary files
127+
emitted and
128+
consumed by the compiler and linker on the way to making the final binaries and libraries).
129+
130+
Change into the build directory and configure the package as follows:
131+
132+
```bash
133+
cmake -S <path_to_source_code> -DCMAKE_INSTALL_PREFIX=<path_to_install_directory> -DOPENCV_EXTRA_MODULES_PATH=../../src/OpenCV/opencv_contrib-4.x/modules -DWITH_TIFF=ON -DCMAKE_BUILD_TYPE=Release
134+
```
135+
136+
Let's break down this command:
137+
138+
- `-S` is where the source code to be compiled it. This directory must contain the `CMakeLists.txt` file.
139+
- `-D<option>=<value>` The options being configured. Most are package-specific (such as `-DOPENCV_EXTRA_MODULES_PATH`
140+
and `-DWITH_TIFF=ON`). Some package-agnostic options (the `-DCMAKE_<option>=<value>`) are elaborated upon below:
141+
- `-DCMAKE_INSTALL_PREFIX=<install_directory>`: This option tells CMake the directory under which to place all the
142+
libraries, binaries, headers and auxiliary files. By default, CMake will try to install to your root system and
143+
this can cause conflicts with the package manager. **It is highly recommended to set this to a well-organized
144+
location.**
145+
- `-DCMAKE_BUILD_TYPE=Release`: This tells CMake to build the package in "Release" mode, i.e. enable optimizations
146+
and disable debug information. **Set this for optimal performance**.
147+
148+
Upon running the above command, CMake will check your system to see if everything is available.
149+
If it complains about not finding something, either install it via the method described in the section above, or build
150+
it from source and include its install prefix in the `CMAKE_PREFIX_PATH` environment variable.
151+
152+
If this step is successful, proceed to building and installing the package.
153+
154+
## Step 4: Build and install the package
155+
156+
To build and install the package, simply run the following from the build directory:
157+
158+
```bash
159+
cmake --build . --target install
160+
```
161+
162+
This will start compilation and linking and install the library to the directory specified in the configuration step.
163+
164+
Compiling is compute intensive and large packages can take a long time (5-6 hours to compile PyTorch on an Orin).
165+
166+
Errors in this step can be dealt with by consulting GitHub issues and searching the internet. Common fixes include
167+
installing a more modern compiler, changing versions of dependencies or changing package source code.
168+
169+
When this step completes, the install directory will contain:
170+
171+
```text
172+
├── bin
173+
├── include
174+
├── lib
175+
├── lib64
176+
└── share
177+
```
178+
179+
Note that multiple packages can install to the same install directory.
180+
181+
## Step 5: Link against the built package
182+
183+
You most likely built this package as a dependency to some other target package you need to build.
184+
185+
While building your target package, you need to inform CMake where to find the dependency you just built.
186+
187+
To achieve this, before configuring the target package, add the install directory of the package you just built to the
188+
`CMAKE_PREFIX_PATH` environment variable, like so in bash:
189+
190+
```bash
191+
export CMAKE_PREFIX_PATH="<install_directory>:$CMAKE_PREFIX_PATH"
192+
```
193+
194+
Here, <install_directory> is the same as specified in the variable
195+
`-DCMAKE_INSTALL_PREFIX` during configuration, which has the structure mentioned above below it.
196+
197+
After setting this environment variable, continue the configure process as usual, and CMake will pick up the dependency
198+
from where you installed it.
199+
200+
While running the library, you might encounter errors about not finding a particular `.so` file. To fix these, simply
201+
add the `<install_directory>/lib/` or the `<install_directory>/lib64/` directory to the `LD_LIBRARY_PATH` environment
202+
variable to enable the Linux Dynamic Linker to find the dynamic libraries where you installed them.
203+
An example of this process is
204+
205+
```bash
206+
export LD_LIBRARY_PATH="<install_directory>/lib:$LD_LIBRARY_PATH"
207+
```
208+
209+
## Summary
210+
211+
This tutorial went over the exact process to build `cmake` enabled projects from source.
212+
213+
The ability to do this reduces your reliance on the package manager and helps you get out of dependency issues.
214+
215+
Moreover, using the above framework you can install and use multiple versions of the same library on your system.
216+
217+
## See Also:
218+
219+
- [A short guide on using CMake](/wiki/programming/cmake/)

‎wiki/programming/index.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,9 @@ This section focuses on **programming techniques, tools, and libraries** commonl
3333
- **[Python Construct Library](/wiki/programming/python-construct/)**
3434
Explores the Python Construct library for building and parsing binary data. Highlights its utility for reliable serial communication in robotics systems. Includes examples of creating structured messages with CRC error-checking.
3535

36+
- **[Building Software from Source](/wiki/programming/build-from-source/)**
37+
Covers the process of configuring, building, and installing libraries from source code. Uses OpenCV as an example to demonstrate obtaining source code, managing dependencies, and configuring builds with CMake.
38+
3639
- **[ROS 2 Yasmin State Machine](/wiki/programming/yasmin-ros2-state-machine/)**
3740
A tutorial on using the Yasmin framework for state machine-based task logic in ROS 2. Covers core fundamentals, installation, and practical usage examples.
3841

0 commit comments

Comments
 (0)