Hacking on rpm-ostree 在 rpm-ostree 上进行黑客攻击

  1. Setting up a development environment
    设置开发环境
    1. Via toolbx/Installing natively
      通过 toolbx/本地安装
    2. Using the fcos-buildroot container
      使用 fcos-buildroot 容器
  2. Building and testing 构建和测试
    1. Baseline build 基线构建
    2. Unit tests 单元测试
    3. Virtualized integration testing
      虚拟化集成测试
      1. Iteration on kola external tests
        在 kola 外部测试上的迭代
      2. vmcheck 虚拟机检查
  3. Testing with a custom libdnf
    使用自定义 libdnf 进行测试
  4. Testing with a custom ostree
    使用自定义 ostree 进行测试
  5. Testing with a custom Rust crate (example: ostree-rs-ext)
    使用自定义的 Rust crate 进行测试 (示例: ostree-rs-ext)
  6. Using GDB with the rpm-ostree daemon
    使用 GDB 与 rpm-ostree 守护程序
    1. Server-side (composes) 服务器端 (composes)
    2. Client-side 客户端
      1. Attaching to the daemon
        连接到守护程序
      2. Attaching to the CLI
        连接到 CLI

Setting up a development environment
设置开发环境

The majority of developers on rpm-ostree build and test it from a toolbox container, separate from the host system. The instructions below may also work when run in a traditional login on a virtual machine, but are less frequently tested.
大多数 rpm-ostree 的开发人员会在与主机系统分开的工具箱容器中构建和测试它。下面的说明也可能在虚拟机上的传统登录中运行时有效,但测试频率较低。

Via toolbx/Installing natively
通过 toolbx/本地安装

When developing either in a toolbx container or natively on your system, you must install all the required dependencies. In the ci/ subfolder, there are scripts that will do this for you:
在工具箱容器中或本地开发时,您必须安装所有必需的依赖项。在 ci/ 子文件夹中,有脚本可以为您执行此操作:

  • To install the build dependencies: ./ci/installdeps.sh and ci/install-cxx.sh
    要安装构建依赖项: ./ci/installdeps.shci/install-cxx.sh
    • Note: This command must be rerun after the dependencies in Cargo.lock change. This will eventually be fixed.
      注意:此命令在 Cargo.lock 中的依赖项更改后必须重新运行。这将最终得到修复。
  • To install the test dependencies:
    安装测试依赖项:

Today rpm-ostree uses cxx.rs, and as of https://github.com/coreos/rpm-ostree/pull/3864 we commit the generated code to git. If you want to regenerate it (particularly when changing it), use ci/install-cxx.sh. Most importantly, it currently must be reinstalled after you run make clean on the project.
今天 rpm-ostree 使用 cxx.rs,截至 https://github.com/coreos/rpm-ostree/pull/3864,我们将生成的代码提交到 git。如果您想重新生成它(特别是在更改它时),请使用 ci/install-cxx.sh 。最重要的是,在运行 make clean 项目后,目前必须重新安装它。

Using the fcos-buildroot container
使用 fcos-buildroot 容器

The fcos-buildroot container has all the dependencies needed for building and testing included. As a consequence, it is comparatively large (~ 2.5 GB). Since this is the same container used by the CI system, this is useful for reproducing CI failures.
fcos-buildroot 容器包含构建和测试所需的所有依赖项。因此,它相对较大(~ 2.5 GB)。由于这是 CI 系统使用的相同容器,因此对于重现 CI 失败很有用。

You can either work from inside the container in an interactive manner:
您可以以交互方式从容器内部工作:

# IMPORTANT: Run this command from the projects root directory!
$ podman run --rm -it -v "$PWD:$PWD:z" -w "$PWD" \
    quay.io/coreos-assembler/fcos-buildroot:testing-devel

Or you use the container in an ephemeral fashion with an alias like this:
或者您可以像这样使用容器以临时方式带有别名:

# IMPORTANT: Run this command from the projects root directory!
$ alias buildroot="podman run --rm -it -v \"$PWD:$PWD:z\" -w \"$PWD\" \
    quay.io/coreos-assembler/fcos-buildroot:testing-devel"
# These commands run in the container now
$ buildroot make ...

The above commands will mount your current working directory into the container for your build artifacts etc. to persist.
上述命令将您当前的工作目录挂载到容器中,以便持久保存构建产物等。

Building and testing 构建和测试

Baseline build 基线构建

rpm-ostree uses autotools to build both our C/C++ side as well as to invoke cargo to build the Rust code. After you’ve cloned the repository:
rpm-ostree 使用 autotools 来构建我们的 C/C++ 代码,同时也用来调用 cargo 来构建 Rust 代码。在克隆了仓库之后:

$ git submodule update --init
$ ./autogen.sh --prefix=/usr --libdir=/usr/lib64 --sysconfdir=/etc
$ make

Unit tests 单元测试

$ make check

Virtualized integration testing
虚拟集成测试

The unit tests today don’t cover much; rpm-ostree is very oriented to run as a privileged systemd unit managing a host system.
今天的单元测试覆盖范围不大;rpm-ostree 非常倾向于作为特权 systemd 单元在管理主机系统时运行。

rpm-ostree has some tests that use the coreos-assembler/kola framework.
rpm-ostree 有一些使用 coreos-assembler/kola 框架的测试。

You will want to build a custom image, then run make install in the ${topsrcdir}/tests/kolainst/ directory, and finally kola run --qemu-image path/to/custom-rpm-ostree-qemu.qcow2 'ext.rpm-ostree.*'. See the kola external tests documentation for more information and also how to filter tests.
您需要构建一个自定义镜像,然后在 ${topsrcdir}/tests/kolainst/ 目录中运行 make install ,最后运行 kola run --qemu-image path/to/custom-rpm-ostree-qemu.qcow2 'ext.rpm-ostree.*' 。查看 kola 外部测试文档以获取更多信息,以及如何过滤测试。

Iteration on kola external tests
kola 外部测试的迭代。

make                                 # To build
cosa build-fast                      # Generate a local FCOS VM
sudo make -C tests/kolainst install  # Run this only when you change the test suite
kola run ext.rpm-ostree.destructive.container-image  # Or any other test you want

vmcheck 虚拟机检查

There’s also a vmcheck test suite, which predates Fedora CoreOS, coreos-assembler and the external test suite integration.
还有一个 vmcheck 测试套件,早于 Fedora CoreOS、coreos-assembler 和外部测试套件集成。

One approach for (somewhat) fast iteration is cosa build-fast, then run e.g. ./tests/vmcheck.sh.
一种(有点)快速迭代的方法是 cosa build-fast ,然后运行例如 ./tests/vmcheck.sh

To filter tests, use the TESTS= environment variable. For example, to run only tests/vmcheck/test-misc-2.sh, you can do:
要过滤测试,请使用 TESTS= 环境变量。例如,要仅运行 tests/vmcheck/test-misc-2.sh ,您可以执行以下操作:

TESTS='misc-2' ./tests/vmcheck.sh

For development, there is also a make vmsync which copies the built rpm-ostree into an unlocked VM. To use this, you must have an ssh-config file with a host defined in it called vmcheck. You can provision the VM however you want; libvirt directly, vagrant, a remote OpenStack/EC2 instance, etc. For QEMU, we have a helper script at tests/vm.sh which uses kola to spawn a CoreOS QEMU image. You can use it like this:
对于开发,还有一个 make vmsync ,它将构建的 rpm-ostree 复制到一个未锁定的虚拟机中。要使用此功能,您必须拥有一个包含主机定义为 vmcheckssh-config 文件。您可以按照自己的喜好配置虚拟机;直接使用 libvirt,vagrant,远程 OpenStack/EC2 实例等。对于 QEMU,我们有一个辅助脚本位于 tests/vm.sh ,它使用 kola 来生成 CoreOS QEMU 镜像。您可以像这样使用它:

export COSA_DIR=/path/to/cosa/workdir
tests/vm.sh spawn
make vmsync

Note that by default, these commands will retrieve the latest version of ostree from the build environment and include those binaries when syncing to the VM. So make sure to have the latest ostree installed or built. This allows you to not have to worry about using libostree APIs that are not yet released.
请注意,默认情况下,这些命令将从构建环境中检索 ostree 的最新版本,并在同步到虚拟机时包含这些二进制文件。因此,请确保安装或构建最新的 ostree。这样可以避免担心使用尚未发布的 libostree API。

For more details on how tests are structured, see tests/README.md.
有关测试结构的更多详细信息,请参阅 tests/README.md。

Testing with a custom libdnf
使用自定义 libdnf 进行测试

rpm-ostree bundles libdnf since commit https://github.com/coreos/rpm-ostree/commit/125c482b1d16ce8376378f220fc2f93a5b157bc1 the rationale is:
自从提交 https://github.com/coreos/rpm-ostree/commit/125c482b1d16ce8376378f220fc2f93a5b157bc1 以来,rpm-ostree 捆绑了 libdnf,其原因是:

  • libdnf broke ABI several times silently in the past
    libdnf 在过去多次悄悄地破坏了 ABI
  • Today, dnf does not actually use libdnf much, which means for the most part any libdnf breakage is first taken by us
    今天,dnf 实际上并没有大量使用 libdnf,这意味着在大多数情况下,任何 libdnf 的破坏都首先由我们承担
  • libdnf is trying to rewrite more in C++, which is unlikely to help API/ABI stability
    libdnf 正试图更多地用 C++重写,这不太可能有助于 API/ABI 的稳定性
  • dnf and rpm-ostree release on separate cycles (e.g. today rpm-ostree is used by OpenShift)
    dnf 和 rpm-ostree 分别发布在不同的周期上(例如,今天 rpm-ostree 被 OpenShift 使用)

In general, until libdnf is defined 100% API/ABI stable, we will continue to bundle it.
一般来说,直到 libdnf 被定义为 100% 的 API/ABI 稳定,我们将继续捆绑它。

However, because it’s a git submodule, it’s easy to test updates to it, and it also means we’re not forking it.
然而,因为它是一个 git 子模块,所以很容易测试更新,这也意味着我们没有对其进行分叉。

So just do e.g.: 所以只需执行例如:

cd libdnf
git fetch origin
git reset --hard origin/main
cd ..

The various make targets will pick up the changes and recompile.
各种 make 目标将捕捉到更改并重新编译。

Testing with a custom ostree
使用自定义 ostree 进行测试。

It is sometimes necessary to develop against a version of ostree which is not even yet in git main. In such situations, one can simply do:
有时需要针对尚未在 git 主分支中的 ostree 版本进行开发。在这种情况下,可以简单地执行以下操作:

$ # from the rpm-ostree build dir
$ INSTTREE=$PWD/insttree
$ rm -rf $INSTTREE
$ # from the ostree build dir
$ make
$ make install DESTDIR=$INSTTREE
$ # from the rpm-ostree build dir
$ make
$ make install DESTDIR=$INSTTREE

At this point, simply set SKIP_INSTALL=1 when running vmsync and vmoverlay to reuse the installation tree and sync the installed binaries there:
在这一点上,只需在运行 vmsyncvmoverlay 时设置 SKIP_INSTALL=1 ,以重用安装树并同步已安装的二进制文件:

$ make vmsync SKIP_INSTALL=1
$ make vmoverlay SKIP_INSTALL=1

Of course, you can use this pattern for not just ostree but whatever else you’d like to install into the VM (e.g. bubblewrap, libsolv, etc…).
当然,您不仅可以将此模式用于 ostree,还可以用于您想要安装到虚拟机中的任何其他内容(例如 bubblewrap、libsolv 等)。

Testing with a custom Rust crate (example: ostree-rs-ext)
使用自定义的 Rust crate 进行测试(示例:ostree-rs-ext)

A common case is testing changes in the ostree-rs-ext crate. Once you have your changes ready in a clone of the ostree-rs-ext repository, you can edit rpm-ostree’s Cargo.toml to point to it.
一个常见情况是测试 ostree-rs-ext crate 中的更改。一旦您在 ostree-rs-ext 存储库的克隆中准备好更改,您可以编辑 rpm-ostree 的 Cargo.toml 来指向它。

...
[patch.crates-io]
ostree-ext = { path = "../ostree-rs-ext/lib/" }

See the example from the Overriding Dependencies section of the Cargo Book.
请参阅 Cargo Book 中覆盖依赖项部分的示例。

Using GDB with the rpm-ostree daemon
使用 GDB 与 rpm-ostree 守护程序

If you’re new to rpm-ostree, before using GDB, it may be helpful to review the daemon architecture doc for an architecture recap.
如果您是 rpm-ostree 的新手,在使用 GDB 之前,可能有必要查看守护程序架构文档,以便回顾架构。

Server-side (composes) 服务器端(构建)

Server-side composes do not use the daemon architecture and so one can naturally do e.g. gdb --args rpm-ostree compose tree .... If using coreos-assembler, you can set the COSA_RPMOSTREE_GDB environment variable like this:
服务器端组合不使用守护程序架构,因此可以自然地执行例如 gdb --args rpm-ostree compose tree ... 。如果使用 coreos-assembler,可以像这样设置 COSA_RPMOSTREE_GDB 环境变量:

$ COSA_RPMOSTREE_GDB="gdb --args" cosa build

When cosa gets to the point of invoking rpm-ostree for the compose, it will call GDB instead.
当 cosa 到达调用 rpm-ostree 进行组合的阶段时,它将调用 GDB 而不是。

Client-side 客户端

On the client side, you need to use the make vmsync flow before using GDB because it also copies over the source files into /root/sync.
在客户端上,在使用 GDB 之前,您需要使用 make vmsync 流程,因为它还会将源文件复制到 /root/sync 中。

You can use GDB from a privileged container. Make sure to use the --pid=host flag when using e.g. podman run so that you can attach to processes running on the host. For example:
您可以从特权容器中使用 GDB。确保在使用例如 podman run 时使用 --pid=host 标志,以便您可以附加到在主机上运行的进程。例如:

(host) podman run -ti --privileged --pid=host -v /:/host --name gdb \
        registry.fedoraproject.org/fedora:36 /bin/bash
(cnt) dnf install -y gdb procps-ng

If for whatever reason, you can’t use a container, you can also layer GDB with e.g. rpm-ostree install gdb-minimal -A, and then use it directly. (XXX: apply-live currently isn’t compatible with make vmsync, so you’ll want to reboot for now: https://github.com/ostreedev/ostree/issues/2369).
如果由于任何原因,您无法使用容器,您还可以将 GDB 与例如 rpm-ostree install gdb-minimal -A 层叠,然后直接使用它。(XXX: apply-live 目前与 make vmsync 不兼容,因此您需要重新启动:https://github.com/ostreedev/ostree/issues/2369)。

Attaching to the daemon
附加到守护程序

Run e.g. rpm-ostree status to ensure the daemon is started, and then:
运行例如 rpm-ostree status 以确保守护程序已启动,然后:

(cnt) gdb -p $(pidof rpm-ostree) \
        -ex 'set sysroot /host' \
        -ex 'directory /host/var/roothome/sync' \
        -ex 'directory /host/var/roothome/sync/libdnf/libdnf'

Then in GDB, you can do e.g.:
然后在 GDB 中,您可以执行例如:

(gdb) break deploy_transaction_execute
(gdb) continue

And in a separate terminal in the VM, run the CLI command which would trigger the breakpoint (e.g. rpm-ostree override replace foobar.rpm).
在虚拟机中的单独终端中运行会触发断点的 CLI 命令(例如 rpm-ostree override replace foobar.rpm )。

Attaching to the CLI
附加到 CLI

To attach to the CLI itself (or debug early daemon startup), you can use gdb --args rpm-ostree status if running GDB from the host directly.
要附加到 CLI 本身(或调试早期守护程序启动),您可以直接从主机运行 GDB 使用 gdb --args rpm-ostree status

If running GDB in a container, you can use the RPMOSTREE_GDB_HOOK env var to have rpm-ostree wait for you to attach GDB from the container:
如果在容器中运行 GDB,则可以使用 RPMOSTREE_GDB_HOOK 环境变量,让 rpm-ostree 等待您从容器中附加 GDB:

(host) RPMOSTREE_GDB_HOOK=1 rpm-ostree status
RPMOSTREE_GDB_HOOK detected; stopping...
Attach via gdb using `gdb -p 2519`.

Then: 然后:

(cnt) gdb -p 2519 \
        -ex 'set sysroot /host' \
        -ex 'directory /host/var/roothome/sync' \
        -ex 'directory /host/var/roothome/sync/libdnf/libdnf' \
        -ex n -ex n