0.0.0
preliminary (some mature, much in progress)
Domain and Tool Landscape
Concepts, Workflow and Build System
Dive into the deep – coming soon
build with Sphinx, presented with Reveal.js
Copyright © 2018‒2021 Li-Pro.Net and individual contributors. — All rights reserved. — CC-BY-SA-3.0 — Release: 0.0.0-75-g2094a49
Involved Components:
Git – Repo and Manifest
Yocto – Terms and Workflow
Bitbake – Layer and Recipies
Audience:
System Architects
Hard- and Software Developers
Integrators and Testers
Revision Controll Systems
Configuration and Build Tools
Distribution and Packaging
Scripting
Git
Subversion
Mercurial
Principals of work
Collaboration work
Kconfig + GNU Make
Autotools and CMake
Qmake
Ninja
Source Code Patching
Triad: config
→ make
→ install
Canadian Cross: build
→ host
→ target
Debian Packages: *.deb
Red Hat Packages: *.rpm
Open Packages: *.opkg
/ *.ipkg
System Root: SYSROOT
Destination Directory: DESTDIR
Package Repository (Server)
Bash
Python
Perl (rarely)
→ that are:
Customer and User
Retailer and Service
→ which are constantly looking for:
Improvements and Bug Fixes (updates)
Features and Extensions (back ports)
Documentation and Tests
→ all participants and partners:
Engineers
Business Owners
Compliance Offices
→ have to ensure results with:
protected release base line: Software Configuration Management Plan
build and test: Continuous Integration
DISTRIBUTION: Continuous Deployment / Delivery
→ what is their role ❓
best efforts to COOPERATE with and MUTUAL ASSISTANCE the business
🈚 free of charge: SCALE, BE FAST, pay later
🈺 open for business: SHARE WITH ALL, no protection, survival of the fittest
🈴 passing grade: BEST FOR ALL, acceptance
㊙ secret: kick out the Troll
→ where and who am I ❓
mostly I am a tiny candle in front of the biggest star of all
leads to: relation of 🕯 candle / ⭐ star
ask you: what is MY BUSINESS 🕯 candle
ask you: what is THE COMMUNITY ⭐ star
my experience of the last 20 years:
10–40 % 🕯 candle / 90–60 % ⭐ star
→ the COMMUNITY – there are we all ‼
Market with its Suppliers and Vendors
Contractors and their Principals
→ we should take over the WORKFLOWS ‼
Infrastructure
Frameworks
Toolboxes
from the perspective of:
Architecture
Life Cycle
1 – Product Management
which hard- and software features we have, and need for a specific product → feature sets
2 – Feature Management
how the distribution looks for a given feature set → composing
3 – Distribution Management
which packages will needed for a given distribution / composing → flavor
4 – Package Management
how to build and deploy the given list of packages and their dependencies
5 – Deployment Management
what have to share and how to distribute all the resulting artefacts
6 – QA Management (Test, Integration)
how to ensure accuracy of all the development, in parts and altogether
excerpts and quotes from [Str15,OMM15]. — © 2015 The Linux Foundation.
Embedded Linux Landscape
Android
Baserock
Buildroot
OpenEmbedded
OpenWrt
The Yocto Project
Commercial Distributions
Reflection
Buildroot ↔ The Yocto Project
Great for systems with ARM-based SoCs and touch screens
Build system and development tools by Google
Java based system runtime environment on target
Stable and widely accepted Java API for final applications
Very limited adaptability for vendors, contributors or customers
Targeted for embedded appliances
Slightly pushed by the GENIVI Alliance
Native Builds for x86, x86_64, ARMv7
Based on Morph workflow tool
Maintained by Codethink Ltd.
Last releases in 2016. → https://wiki.baserock.org/releases/
→ https://www.openembedded.org/
Created by merging OpenZaurus with contributions from Familiar Linux, OpenSIMPad into a common code base
Focusing on broad hardware and architectures
Based on BitBake build engine
Technology code base for the Yocto Project and the Ångström Distribution
The Ångström Distribution was widely used on TI-based embedded boards like the BeagleBoard and PandaBoard and uses the Yocto Project build environment. Last releases in 2017 (2020). → https://github.com/Angstrom-distribution
Debuted as open source OS for embedded devices routing network traffic
Originally created from Linksys GPL sources for their WRT54G residential gateway
Targeted thousands of different hardware
Buildroot-based build environment
Headless operation with web UI
Member of Software Freedom Conservancy (like Git, Inkscape, or Qemu)
→ https://lwn.net/Articles/682540/ (ELCE-2016, ELCE-2018, Youtube)
Buildroot |
Yocto Project |
|
---|---|---|
Minimal Size: |
2.2 MB |
4.9 MB |
Build Time: |
15-45 minutes |
1-2 hours (at least) |
Concept: |
single Makefile |
layered classes |
Complexity: |
being simple by design, only patchable |
multiple aspects of composing and QA |
Composing: |
multiple Kconfig file |
feature driven |
State Cache: |
compiler only |
on task level (states) |
Buildroot |
Yocto Project |
|
---|---|---|
Rebuild: |
full (will be improved) |
partial per task / on changes |
Meta/Configure: |
fixed |
sharable |
Output: |
no packages or SDK |
images, packages, SDK, licenses, manifests |
Reliance: |
no QA workflow |
documented QA and RM |
3rd Party: |
rare, some few |
many (growing) |
Decision: |
Buildroot |
Yocto Project |
The Yocto Project is not an Embedded Linux Distribution.
It creates a custom one for You!
The Yocto Project is not Single Open Source Project.
It is an Ecosystem.
The Yocto Project combines the convenience of a ready-to-run Linux Distribution with the flexibility of a custom Linux operating system stack. [Str15,OMM15] — © 2015 The Linux Foundation and Yocto Project.
based on meritocracy (leadership by demonstrated achievement)
managed by its chief architect, Richard Purdie, a Linux Foundation fellow
remain independent of any one of its member organizations
The Yocto Project is a lab workgroup of the Linux Foundation, who owns its trademark.
open source collaboration project
founded in 2010 as a collaboration among many hardware manufacturers, open-source operating systems vendors and electronics companies
help you create custom Linux-based systems for embedded products
regardless of the hardware architecture
provides templates, tools and methods (Best-Practice)
resources and information for new and experienced users
core system component recipes by OE
example code to demonstrate its capabilities
include the Yocto Project Linux kernel
covers several build profiles across multiple architectures: ARM, PPC, x86, and more
BSP layers for customer or vendor specific platform support
BSP layers follow a predetermined and standardized format
embedded tools
best practices
reference implementation
Linux Foundation: chief architect
OpenEmbedded: technologies and software
silicon vendors and electronics companies
slightly extended
Git Repo (repo
):
The collection manager that Google has built on top of Git to help
manage multiple Git repositories, does the uploads to
Gerrit and automates the Android system image development workflow.
The repo command is an executable Python script that you
can put anywhere in your path.
→ https://gerrit.googlesource.com/git-repo/
BitBake: The Task executor and scheduler used by the Build System to build Images. The bitbake command is part of an fancy Python and Bash environment around the BitBake project. → BitBake, or Yocto Components and Tools → BitBake, or BitBake User Manual
Metadata: The files that BitBake parses when building an Images. In general, Metadata provides Recipes, Classes and Configuration Files. → Metadata, or BitBake User Manual → Overview → Introduction
OE-Core: A core set of Metadata originating with OpenEmbedded that is shared between OE and the Yocto Project. → OpenEmbedded-Core (OE-Core), or Yocto Components and Tools → OE-Core, or OE Wikipedia
Poky: The reference distribution of the Yocto Project. It contains the Build System (BitBake and OE-Core) as well as a set of Metadata to get you started building your own distro. → Poky, or Yocto Components and Tools → Poky, or Yocto Reference Distribution
Layer: A collection of Recipes representing the core, a Board Support Package, an application stack, or any other 3rd party contributions. Layers are hierarchical in their ability to override previous specifications. → Layer, or in Concept
list of Yocto Compatible Layers
list of OpenEmbedded Contributed Layers
Image: An artifact of the BitBake build process given a collection of Recipes and related Metadata. Images are the binary output that run on specific target architecture. → Image, or in Concept, or in Reference
Package: Refers to a Recipe’s packaged output produced by BitBake (i.e. a “baked recipe”). A Package is generally the compiled binaries produced from the Recipe’s sources. You “bake” any “bin” by running it through BitBake. → Package
Package Groups: Arbitrary group of software Recipes. You use Package Groups to hold Recipes that, when built, usually accomplish a single Task. For example, Package Groups could contain the Recipes for a company’s proprietary or value-add software. → Package Groups
Task: A unit of execution for BitBake (e.g. do_fetch, do_patch, do_compile, and so forth). → Task, or in Reference
Classes:
Files with the .bbclass
file extension that provided for
logic encapsulation and inheritance so that commonly
used patterns can be defined once and then easily used
in multiple Recipes.
→ Classes, or in
Concept, or in
Reference
Recipes:
Files with the .bb
file extension have a set of
instructions for building Packages or Package
Groups or Images and describes where you get source code,
which patches to apply, how to configure the source, how
to compile it and so on. It also describe dependencies
for libraries or for other Recipes. Recipes
represent the logical unit of execution.
→ Recipe, or in
Concept
Append Files:
Files with the .bbappend
file extension that append build
information to a Recipe. The Build System respects
every Append file to have a corresponding Recipe
(.bb
) file.
→ Append Files
Configuration Files: Configuration information in various → Configuration File, or in Concept
Features provide a mechanism for:
working out which Packages or Package Groups should be included in the generated Image, typically configured in Image Recipes and their related Classes → Image Features
distributions Configuration Files can select which features they want to support → Distro Features
machine Configuration Files specifies the hardware functions that will be available for a given machine → Machine Features
backward and inter-layer compatibility for special (historical) cases → Feature Backfilling
Build System: The build system specific to the Yocto Project based on the project Poky, which uses BitBake as the Task executor.
Source Directory: Refers to the directory structure created as a result of creating a local copy of the Poky and all other Git repositories as referred by the Git Repo manifest. It contains BitBake, Documentation, Metadata and other files that all support the Yocto Project. Consequently, you must have the Source Directory in place on your development system in order to do any development using the Yocto Project. → Source Directory
Build Directory: This term refers to the area used by the Build System for downloads, builds and deployments. → Build Directory
Cross-Development Toolchain: A collection of cross-compilers, linkers, debuggers, libraries and utilities that run on one architecture and allow you to develop software for a different, or targeted, architecture. → Cross-Development Toolchain
Board Support Package (BSP): A group of drivers, definitions, and other components that provide support for a specific hardware configuration. → Board Support Package (BSP), or Yocto Project Board Support Package Developer’s Guide and Yocto Project Linux Kernel Development Manual
the following applies and is unavoidable
Git Repo: determines the code base line
Layers: are essential and able to overlay
Classes: inheritable but not extensible
Recipes: extensible but not inheritable
Features: determines the final product
PREFACE
A product firmware stack will developed locally and pushed to different in-house or external repositories at irregular intervals. A tool to unifies all that repositories as necessary, performs uploads and automates the pull and sync process will be needed.
The repo tool and a meta Git repository called manifest or such as in our case platform repo make it possible to get all the code needed to build your own Yocto source and build directory.
excerpts and quotes from https://source.android.com/setup/develop#repo — © 2008–2021 Google LLC.
Git
handle projects that are distributed over multiple Git repositories
Repo
unifies Git repositories, performs uploads to Gerrit, and automates the workflow
Gerrit
code review system for projects that use Git
https://github.com/lipro-yocto/lpn-central-repo
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<default sync-j="4" revision="master"/>
<remote fetch="https://git.yoctoproject.org/git" name="yocto"/>
<remote fetch="https://github.com/openembedded" name="oe"/>
<remote fetch="https://github.com/meta-qt5" name="qt5"/>
<remote fetch="https://github.com/lipro-yocto" name="lpy"/>
<project remote="yocto" name="poky" path="sources/poky"/>
<project remote="yocto" name="meta-mingw" path="sources/yocto/meta-mingw"/>
<project remote="oe" name="meta-openembedded" path="sources/openembedded"/>
<project remote="qt5" name="meta-qt5" path="sources/qt/meta-qt5" />
<project remote="lpy" name="lpn-central" path="sources/central">
<copyfile dest="setup-environment" src="scripts/setup-environment"/>
</project>
</manifest>
OpenEmbedded Build System Concepts
In general, the build’s workflow consists of several functional areas.
Setup values you can use to control the build process.
Various layers that provide software, machine and distro metadata.
Upstream releases, local projects, and SCMS.
Directories containing output packages (RPM, DEB or IPK), which are subsequently used in the construction of an image or SDK.
Processes under the control of BitBake. This block expands on how BitBake fetches source, applies patches, completes compilation, analyzes output for package generation, creates and tests packages, generates images and generates cross-development tools.
Images produced by the build system.
Cross-development tools which are produced with the build system.
Through user configuration you can tell BitBake the target architecture for which you are building the image, where to store downloaded source and other build properties.
bblayers.conf
https://github.com/lipro-yocto/lpn-central → meta-cedi/conf
CEDI_BBLAYERS_CONF_VERSION = "1"
BBPATH = "${TOPDIR}"
BSPDIR := "${@os.path.abspath('##OEROOT##/../..')}"
BBFILES ?= ""
BBLAYERS = " \
${BSPDIR}/sources/poky/meta \
${BSPDIR}/sources/poky/meta-poky \
${BSPDIR}/sources/poky/meta-yocto-bsp \
\
${BSPDIR}/sources/yocto/meta-mingw \
\
${BSPDIR}/sources/openembedded/meta-oe \
${BSPDIR}/sources/openembedded/meta-python \
${BSPDIR}/sources/openembedded/meta-initramfs \
${BSPDIR}/sources/openembedded/meta-filesystems \
${BSPDIR}/sources/openembedded/meta-networking \
${BSPDIR}/sources/openembedded/meta-multimedia \
\
${BSPDIR}/sources/qt/meta-qt5 \
\
${BSPDIR}/sources/central/meta \
${BSPDIR}/sources/central/meta-cedi \
${BSPDIR}/sources/central/meta-lpn-bsp \
"
The bblayers.conf
file tells BitBake what layers you want
considered during the build.
local.conf
https://github.com/lipro-yocto/lpn-central → meta-cedi/conf
The local.conf
file provides many basic variables that define
a build environment. Here is a list of a few:
Target Machine Selection:
MACHINE ??= "qemux86-64"
Target Distribution Selection:
DISTRO ?= "cedi"
Target Packaging Format:
PACKAGE_CLASSES ?= "package_rpm"
Download Directory:
DL_DIR ?= "${TOPDIR}/downloads"
Shared State Cache Directory:
SSTATE_DIR ?= "${TOPDIR}/sstate-cache"
Build Output:
TMPDIR = "${TOPDIR}/tmp"
Parallelism Options:
BB_NUMBER_THREADS ?= "4"
Host SDK Machine Selection:
SDKMACHINE ?= "x86_64"
The setup-environment script have to use to generate the
content of the local.conf
file.
setup-environment
https://github.com/lipro-yocto/lpn-central → scripts
The setup-environment script provides many basic options that setup a build environment with the correct User Configuration. Here is a short synopsis:
DISTRO="cedi" \
MACHINE="helium" \
SDKMACHINE="x86_64-mingw32" \
BUILD_DIR="build-cedi-helium" \
source setup-environment
The setup-environment calls the oe-init-build-env script implicitly from the correct place in the Poky Source Directory.
To get a detailed online help use:
source setup-environment --help
These layers provide Metadata for the software, machine and policy.
Software Layers: Metadata .bb + Patches
user/vendor/community supplied recipe files, patches and append files
Distribution Layers: Policy Configuration
top-level or general policies for Images or SDKs being built
BSP Layers: Machine Configuration
specific to a particular target architecture
Metadata Layer Priorities (BBFILE_PRIORITY_LayerName
) – remember:
https://github.com/lipro-yocto/lpn-central → meta-lpn-bsp/conf/machine
The meta-lpn-bsp
layer provides machine configurations for all the
specific evaluation hardware based on QEMU or real target architecture. The
following target machines are supported:
hydrogen:
QEMU x86 machine based on qemux86
helium:
QEMU x86-64 machine based on qemux86-64
#@TYPE: Machine
#@NAME: QEMU x86 machine
#@DESCRIPTION: Machine configuration for running an x86 system on QEMU (based on qemux86).
#@MAINTAINER: Stephan Linz <linz@li-pro.net>
require conf/machine/include/qemux86.inc
require conf/machine/include/lpn-emu.inc
qemux86.inc
# derived from qemux86
require conf/machine/qemux86.conf
MACHINEOVERRIDES_prepend = "qemux86:"
# derived this machine back to qemux86 for the Linux kernel
KMACHINE = "qemux86"
lpn-emu.inc
# is a Li-Pro.Net emulated system
require conf/machine/include/lpn.inc
MACHINEOVERRIDES_append = ":lpnemu"
lpn.inc
# is a Li-Pro.Net system
MACHINEOVERRIDES_append = ":lpn"
#@TYPE: Machine
#@NAME: QEMU x86-64 machine
#@DESCRIPTION: Machine configuration for running an x86-64 system on QEMU (based on qemux86-64).
#@MAINTAINER: Stephan Linz <linz@li-pro.net>
require conf/machine/include/qemux86-64.inc
require conf/machine/include/lpn-emu.inc
qemux86-64.inc
# derived from qemux86-64
require conf/machine/qemux86-64.conf
MACHINEOVERRIDES_prepend = "qemux86-64:"
# remap derived machine back to qemux86-64 for the Linux kernel
KMACHINE = "qemux86-64"
lpn-emu.inc
# is a Li-Pro.Net emulated system
require conf/machine/include/lpn.inc
MACHINEOVERRIDES_append = ":lpnemu"
lpn.inc
# is a Li-Pro.Net system
MACHINEOVERRIDES_append = ":lpn"
https://github.com/lipro-yocto/lpn-central → meta-cedi/conf/distro
The meta-cedi
layer provides policy configurations for all the
specific evaluation distributions based on Poky. The following distributions
are supported:
cedi:
Central Distro (Li-Pro.Net Demonstration) based on poky
require ${COREBASE}/meta-poky/conf/distro/poky.conf
DISTROOVERRIDES = "poky:${@d.getVar('DISTRO') or ''}"
DISTRO = "cedi"
DISTRO_NAME = "Central Distro (Li-Pro.Net Demonstration)"
DISTRO_VERSION_append = "+r1.0"
DISTRO_CODENAME_append = "-cedi"
TARGET_VENDOR = "-cedi"
SDK_VENDOR = "-cedisdk"
MAINTAINER = "Cedi <cedi@li-pro.net>"
LOCALCONF_VERSION = "1"
# Override poky pre-settings (remove).
POKY_DEFAULT_DISTRO_FEATURES = ""
POKY_DEFAULT_EXTRA_RDEPENDS = ""
POKY_DEFAULT_EXTRA_RRECOMMENDS = ""
# Setup Cedi defaults.
CEDI_DEFAULT_DISTRO_FEATURES = "largefile opengl ptest multiarch wayland vulkan"
CEDI_DEFAULT_EXTRA_RDEPENDS = "packagegroup-core-boot"
CEDI_DEFAULT_EXTRA_RRECOMMENDS = "kernel-module-af-packet"
DISTRO_FEATURES = "${DISTRO_FEATURES_DEFAULT} ${CEDI_DEFAULT_DISTRO_FEATURES}"
DISTRO_EXTRA_RDEPENDS += " ${CEDI_DEFAULT_EXTRA_RDEPENDS}"
DISTRO_EXTRA_RRECOMMENDS += " ${CEDI_DEFAULT_EXTRA_RRECOMMENDS}"
# Use systemd for system initialization
DISTRO_FEATURES_append = " systemd"
VIRTUAL-RUNTIME_init_manager = "systemd"
VIRTUAL-RUNTIME_login_manager = "shadow-base"
DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit"
VIRTUAL-RUNTIME_initscripts = "systemd-compat-units"
# Add Cedi sanity bbclass.
INHERIT += "cedi-sanity"
https://github.com/lipro-yocto/lpn-central → meta/recipes-core/images
The meta
(central core) layer provides recipes and classes for all the
specific evaluation and demonstration images and package groups based on the
customized image class central-image.bbclass
. The following images are
supported:
central-image-minimal: A small image just capable of allowing an Central device to boot.
central-dev-image: A developer image just capable of allowing an Central device to boot.
central-debug-image: A debugging image just capable of allowing an Central device to boot.
central-image: A product image capable of allowing an Central device to boot and provides full feature support.
require central-image-base.bb
SUMMARY = "A product image capable of allowing an Central \
device to boot and provides full feature support."
DESCRIPTION = "A product image capable of allowing an Central \
device to boot in graphical mode and provides full feature support."
LICENSE = "MIT"
PR = "r0"
IMAGE_FEATURES += " \
${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'x11-base', '', d)} \
"
inherit central-image
central-image-base.bb
SUMMARY = "A console-only image that fully supports the target device \
hardware provided by Central."
LICENSE = "MIT"
PR = "r0"
IMAGE_FEATURES += " \
package-management \
splash \
"
inherit central-image
# Common code for generating Central reference images
LIC_FILES_CHKSUM = "\
file://${CENTRALCOREBASE}/LICENSE;md5=4d183b8707e22082e5a8c5ad268e5149 \
"
# - qt5-sdk - Qt5/X11 SDK and demo applications
# - tools-cross - tools usable for basic cross development
FEATURE_PACKAGES_qt5-sdk = "\
packagegroup-qt5-toolchain-target \
packagegroup-qt5-qtcreator-debug \
"
FEATURE_PACKAGES_tools-cross = ""
# Provides the Central specific features 'qt5-sdk' and 'tools-cross'.
CENTRAL_IMAGE_BASE_INSTALL = '\
packagegroup-central-boot \
packagegroup-base-central \
\
${@bb.utils.contains("IMAGE_FEATURES", "tools-cross", \
"packagegroup-central-tools-cross", "", d)} \
${@bb.utils.contains("IMAGE_FEATURES", "tools-debug", \
"packagegroup-central-tools-debug", "", d)} \
${@bb.utils.contains("IMAGE_FEATURES", "tools-profile", \
"packagegroup-central-tools-profile", "", d)} \
${@bb.utils.contains("IMAGE_FEATURES", "tools-testapps", \
"packagegroup-central-tools-testapps", "", d)} \
${@bb.utils.contains("IMAGE_FEATURES", "tools-sdk", \
"packagegroup-central-sdk", "", d)} \
\
${CENTRAL_IMAGE_EXTRA_INSTALL} \
'
CENTRAL_IMAGE_EXTRA_INSTALL ?= ""
CORE_IMAGE_EXTRA_INSTALL += "${CENTRAL_IMAGE_BASE_INSTALL}"
inherit core-image central-image-version
# Common code for generating Central version file
LIC_FILES_CHKSUM = "\
file://${CENTRALCOREBASE}/LICENSE;md5=4d183b8707e22082e5a8c5ad268e5149 \
"
CENTRAL_VERSION_FILE = "${IMAGE_ROOTFS}${sysconfdir}/central_version"
write_central_version() {
cat > ${CENTRAL_VERSION_FILE} <<EOF
[build information]
vendor-id=LPN
manufacturer-name=Li-Pro.Net
device-variant=${MACHINE}
purpose=${IMAGE_BASENAME}
feature=${IMAGE_FEATURES}
build-number=${BUILD_NUMBER}
EOF
}
ROOTFS_POSTPROCESS_COMMAND += "write_central_version;"
SUMMMARY = "Central Minimal Boot Requirements"
DESCRIPTION = "The minimal set of packages required to boot a Central system"
LICENSE = "MIT"
PR = "r0"
# For backwards compatibility after rename
RPROVIDES_packagegroup-central-boot = "packagegroup-boot-central"
RREPLACES_packagegroup-central-boot = "packagegroup-boot-central"
RCONFLICTS_packagegroup-central-boot = "packagegroup-boot-central"
# packages which content depend on MACHINE_FEATURES need to be MACHINE_ARCH
PACKAGE_ARCH = "${MACHINE_ARCH}"
inherit packagegroup
# Set by the machine configuration with packages essential
# for Central device bootup.
MACHINE_ESSENTIAL_EXTRA_CENTRAL_RDEPENDS ?= ""
MACHINE_ESSENTIAL_EXTRA_CENTRAL_RRECOMMENDS ?= ""
RDEPENDS_${PN} = " \
${MACHINE_ESSENTIAL_EXTRA_CENTRAL_RDEPENDS} \
"
RRECOMMENDS_${PN} = " \
${MACHINE_ESSENTIAL_EXTRA_CENTRAL_RRECOMMENDS} \
"
SUMMMARY = "Central Testing tools/applications"
DESCRIPTION = "The testing set of packages required for a Central system"
LICENSE = "MIT"
PR = "r0"
inherit packagegroup
# Event device test tools
EVTOOLS = "evtest"
# Simple memmory access tools
MEMTOOLS = "devmem2 libuio"
# Simple serial bus tools
SERBUSTOOLS = "i2c-tools spitools"
# Requires Serial to work
USE_SERIAL = "picocom serial-forward"
RDEPENDS_${PN} = " \
${EVTOOLS} \
${MEMTOOLS} \
${SERBUSTOOLS} \
${@bb.utils.contains('MACHINE_FEATURES', 'serial', '${USE_SERIAL}', '',d)} \
"
RRECOMMENDS_${PN} = " \
"
In order for the build system to create an image or any target, it must be able to access source files. The Yocto Project represents source files over the Source Material using Upstream Project Releases, Local Projects, and SCMs.
(almost all recipes)
exist anywhere in the form of an archived file (e.g. tarball or ZIP file)
correspond to individual recipes, PN and PV match with a upstream archive file
archive file can be for any released product that can be built using a recipe
Example
The tslib_1.11.bb
recipe points to the following XZ compressed
tarball file:
https://github.com/kergoth/tslib/releases/download/1.11/tslib-1.11.tar.xz
(no preference for this)
custom bits of software the user provides
these bits reside somewhere local to a project; perhaps a directory into which the user checks in items
use the externalsrc.bbclass class in recipes to include a local projects
Note
Community does not use local projects! Never seen!
(almost vendor or product recipes)
get source files through an SCMS such as Git or Subversion or Mercurial
repository is cloned and/or checked out
Example
The meta/conf/layer.conf
file may tells BitBake how to reach special
(perhaps hidden) Git, Subversion, or Mercurial servers:
CENTRAL_GIT ?= "git://10.20.30.40/services/git"
CENTRAL_SVN ?= "svn://10.20.30.40/services/svn"
CENTRAL_HG ?= "hg://10.20.30.40/services/hg"
(almost all recipes by policy)
The meta-poky/conf/distro/poky.conf
file as required by
the meta-cedi/conf/distro/cedi.conf
file provides
the two kinds of mirrors:
PREMIRRORS
: the pre-mirrors
check before looking Upstream Project Releases
MIRRORS
: the regular mirrors
alternative location should the Upstream Project Releases not available
When the build system generates an image or an SDK, it gets the packages from a package feed area, an intermediary step in the build process. Before placing the packages into package feeds, the build process validates them.
http://git.yoctoproject.org/cgit/cgit.cgi/poky
→ meta/classes/*.bbclass
1. Source Fetching tasks: do_fetch → do_unpack
2. Patch Application tasks: do_patch
3. Configure / Compile / Install tasks: do_configure → do_compile → do_install
4. Analysis / Splitting Packages tasks: do_package/data → do_package_write_* → do_populate_sysroot
5. Image Generation tasks: do_rootfs → do_image/_complete
6. SDK Generation tasks: do_populate_sdk/_ext
tasks by classes: |
cmake |
base |
package/group |
image |
populate_sdk_base |
---|---|---|---|---|---|
do_fetch, do_unpack |
X |
X |
|||
do_patch |
X |
X |
|||
do_configure, do_compile, do_install |
X |
X |
|||
do_package/data, do_package_write_*, do_populate_sysroot |
X |
X |
|||
do_rootfs, do_image/_complete |
X |
||||
populate_sdk/_ext |
X |
The first stages of building a recipe are to fetch and unpack the source code. The do_fetch and do_unpack tasks fetch the source files and unpack them into the Build Directory. BitBake uses the SRC_URI variable to point to source files regardless of their location.
https://github.com/lipro-yocto/meta-lpn-apps (dummy, not yet)
The meta-lpn-apps
layer may provides private (closed) recipes.
The following related SRC_URI may being used:
lib-utility_hg.bb:
"${CENTRAL_HG};
module=lib_utility;
branch=default;
rev=${SRCREV};
protocol=ssh"
lib-crypto_hg.bb:
"${CENTRAL_HG};
module=lib_crypto;
branch=default;
rev=${SRCREV};
protocol=ssh"
app-mcu-ctrl_hg.bb:
"${CENTRAL_HG};
module=app_mcu_ctrl;
branch=default;
rev=${SRCREV};
protocol=ssh"
boost_1.75.0.bb:
"https://dl.bintray.com/boostorg/release/
${PV}/
source/
${BOOST_P}
.tar.bz2"
For private recipes: SRCREV is set to SRCREV = "default"
and CENTRAL_HG
should comes from e.g. meta/conf/layer.conf
.
Once source code is fetched and unpacked, BitBake locates patch files and applies them to the source files. The do_patch task processes recipes by using the SRC_URI variable to locate applicable patch files. BitBake finds and applies multiple patches for a single recipe.
http://git.yoctoproject.org/cgit/cgit.cgi/poky
http://cgit.openembedded.org/meta-openembedded
The poky/meta
and meta-openembedded/meta-oe
layer provides
support recipes. The following related SRC_URI being used:
boost_1.75.0.bb:
SRC_URI += "file://boost-CVE-2012-2677.patch ..."
opencv_4.5.0.bb:
SRC_URI = "git://github.com/opencv/opencv.git;name=opencv ... file://download.patch ..."
Files in the SRC_URI list variable with the .patch
or .diff
extension are automatically identified as a patch and
applied if present beside the recipe.
After source code is patched, BitBake executes tasks that configure and compile the source code. This step consists of three tasks: do_configure, do_compile and do_install. It starts at the earliest after all dependencies from the DEPENDS variable in the recipe have been resolved.
https://github.com/lipro-yocto/meta-lpn-apps (dummy, not yet)
The meta-lpn-apps
layer may provides private (closed) recipes.
The following related DEPENDS may being used:
lib-utility_hg.bb:
"boost"
lib-crypto_hg.bb:
"boost opencv"
app-mcu-ctrl_hg.bb:
"boost opencv qtbase qtsvg lib-utility lib-crypto"
boost_1.75.0.bb:
"boost-build-native zlib bzip2"
opencv_4.5.0.bb:
"libtool swig-native bzip2 zlib glib-2.0 libwebp"
For the application: RDEPENDS may be set to additional
dependent package names that have to be installed later into targets’
runtime environment, e.g. RDEPENDS = "app-observer-daemon"
.
After compilation, the build system analyzes the results and splits the output into packages. The do_package and do_packagedata tasks combine to analyze the files found in the installation directory: debugging symbols, looking at shared library dependencies and relationships of RDEPENDS in the recipe.
https://github.com/lipro-yocto/meta-lpn-apps (dummy, not yet)
The meta-lpn-apps
layer may provides private (closed) recipes.
The following related RDEPENDS may being used:
app-mcu-ctrl:
"i2c-tools"
packagegroup-lpn-apps-toolchain-target:
"boost-staticdev opencv-staticdev lib-utility-staticdev lib-crypto-staticdev"
The following related RDEPENDS may being used as policy;
mostly controlled by common Image Features, e.g.
IMAGE_FEATURES += "tools-testapps"
:
packagegroup-central-tools-testapps:
"devregs spitools v4l-utils"
hostapd_%:
"lpn-mcu-ssid"
Once packages are split and stored in the Package Feeds, the build system uses BitBake to generate the root filesystem image. The do_rootfs task creates the root filesystem for an image.
The final stages of the do_rootfs task handle post processing, includes creation of a manifest file and optimizations. After this, processing begins on the image through the do_image task and dynamically created tasks as needed by the image types; ends with do_image_complete.
The build system uses BitBake to generate the Software Development Kit (SDK) installer script for both the Standard SDK (do_populate_sdk) and Extensible SDK (do_populate_sdk_ext). Like image generation, the SDK script process consists of several stages and depends on many variables.
The images produced by the build system are compressed forms of the root filesystem that are ready to boot on a target device. Images are written out to the deploy directory.
written out to the Build Directory inside the
tmp/deploy/images/machine/
folder for the types:
bootloaders
bootloader binary file, if applicable to the target machine
kernel-image
kernel binary file with specifyed naming scheme
kernel-modules (legacy purposes and can be suppressed)
tarball file with all the modules built for a kernel
root-filesystem-image
root filesystem image files for the target machine
symlinks
symbolic link that points to the most recently built file for each machine
The SDK produced by the build system is the compressed form of a self-extracting installer, which consists of a cross-development toolchain, a set of libraries and headers and an SDK environment setup script. SDKs are written out to the deploy directory.
written out to the Build Directory inside the
tmp/deploy/sdk/
folder for two different types:
Standard SDK: bitbake -c populate_sdk imagename
standard (legacy) SDK w/o any binding into the build system
Extensible SDK: bitbake -c populate_sdk_ext imagename
extensible (next) SDK with complete binding into the build system
Note
An Extensible SDK requires a shared infrastructure!
The Canadian Cross (reduced for host and target only)
Documentations
Presentations
Books
The project documents: https://www.yoctoproject.org/documentation
The project classroom: https://www.yoctoproject.org/learn
The project Wiki-pages: https://wiki.yoctoproject.org/
Bootlin (former Free Electrons) Material: https://bootlin.com/docs/
Training Materials: https://bootlin.com/doc/training/yocto/
Conference Materials: https://bootlin.com/pub/conferences/
Jeffrey Osier-Mixon, Kevin McGrath: Yocto Project and Embedded OS (2015) [OMM15]
Rudolf J. Streif: Introduction to the Yocto Project (2015) [Str15] 📖 [Str16]
Otavio Salvador: Yocto Training : Your next development platform for Embedded Linux (2013) [Sal13] 📖 [SA17]
Alex González: Introduction to Yocto (2012) [Gon12] 📖 [Gon18,VGS16]
Mark Hatle, Bruce Ashfield: Introduction to the Yocto Project (2012) [HA12]
Yi-Hsiu Hsu: Yocto Project Introduction (2015) [Hsu15]
Yen-Chin Lee: Build your own Embedded Linux distributions by Yocto Project (2015) [Lee15]
Nicolas Dechesne, Riku Voipio, Trevor Woerner: OpenEmbedded and Yocto Introduction (2013) [DVW13]
Marcelo A.L. Sanz: Yocto Project Open Source Build System and Collaboration Initiative (2012) [San12]
2016 (480 pages)
|
2017 (162 pages)
|
2017 (478 pages)
|
2018 (456 pages)
|
2016 (989 pages)
|
2016 (214 pages)
|
2015 (144 pages)
|
TODO: complete chapter