|
| 1 | +--- |
| 2 | +name: package |
| 3 | +description: Rebuild the SkyWalking distribution and OAP Docker image after source changes. Use before running e2e tests so the image reflects your code changes. Avoids the "image looks updated but runtime has stale jars" trap. |
| 4 | +argument-hint: "[oap|all|dist-only]" |
| 5 | +--- |
| 6 | + |
| 7 | +# Package OAP Distribution & Docker Image |
| 8 | + |
| 9 | +Rebuilding the Docker image after a source-code change has a **two-step dependency**: rebuild the dist tarball, then rebuild the image. Skipping the first step silently produces an image with stale jars — the Docker build still "succeeds," the image has a fresh timestamp, but the embedded jar is the one from your previous build. |
| 10 | + |
| 11 | +## Why this matters |
| 12 | + |
| 13 | +`make docker.oap` in the root `Makefile` is defined as: |
| 14 | + |
| 15 | +``` |
| 16 | +docker.% push.docker.%: $(CONTEXT)/$(DIST) $(SW_ROOT)/docker/%/* |
| 17 | + $(DOCKER_RULE) |
| 18 | +``` |
| 19 | + |
| 20 | +It depends on `$(CONTEXT)/$(DIST)` (the dist tarball) **as a file prerequisite**. If that tarball already exists, make does not regenerate it — it just copies whatever is on disk into `dist/docker_build/oap/` and runs `docker buildx build`. There is **no** dependency that rebuilds the tarball from source. |
| 21 | + |
| 22 | +Only `make docker` triggers the full chain (`init → build.all → docker.all`). So: |
| 23 | + |
| 24 | +| Command | Rebuilds source? | Rebuilds tarball? | Rebuilds image? | |
| 25 | +|---------|:--:|:--:|:--:| |
| 26 | +| `./mvnw -pl <module> package` | only that module | no | no | |
| 27 | +| `./mvnw -pl apm-dist -am package -Pbackend,dist` | all backend deps + dist | yes | no | |
| 28 | +| `make docker.oap` | **no** | **no** (uses whatever's on disk) | yes | |
| 29 | +| `make docker` | yes (`build.all`) | yes | yes (`docker.all`) | |
| 30 | + |
| 31 | +## `flatten:flatten` — always run it before `package`/`install` |
| 32 | + |
| 33 | +SkyWalking's poms use `${revision}` as a placeholder version (e.g., `10.5.0-SNAPSHOT`). The `flatten-maven-plugin` resolves `${revision}` into concrete versions and writes a `.flattened-pom.xml`. Without it: |
| 34 | + |
| 35 | +- Installed artifacts carry the literal string `${revision}` in their coordinates and cannot be resolved as dependencies. |
| 36 | +- Downstream modules (including `apm-dist`) see an inconsistent dependency graph. |
| 37 | +- Symptoms are subtle: `-pl <module> -am` may succeed in isolation but fail or pull in stale transitive artifacts when the same module is consumed by another build invocation in the same session. |
| 38 | + |
| 39 | +**Always run `flatten:flatten` in the same goal chain as `package` or `install`.** The CI `dist-tar` job and the `compile` skill both do this. Example: |
| 40 | + |
| 41 | +```bash |
| 42 | +./mvnw clean flatten:flatten package -Pbackend,dist -DskipTests |
| 43 | +``` |
| 44 | + |
| 45 | +Not optional — treat it as part of `package`. |
| 46 | + |
| 47 | +## Pick the right command |
| 48 | + |
| 49 | +### Changed OAP source → want a new image |
| 50 | + |
| 51 | +```bash |
| 52 | +# Recommended: full chain |
| 53 | +make docker |
| 54 | +``` |
| 55 | + |
| 56 | +or, faster and equivalent for backend-only changes (skips UI): |
| 57 | + |
| 58 | +```bash |
| 59 | +./mvnw -pl apm-dist -am -o clean flatten:flatten package -Pbackend,dist -DskipTests -Dcheckstyle.skip=true -Dmaven.javadoc.skip=true |
| 60 | +make docker.oap |
| 61 | +``` |
| 62 | + |
| 63 | +**Do not** just run `make docker.oap` after a code edit — the image will look rebuilt but your change will not be in the jars. |
| 64 | +**Do not** skip `flatten:flatten` — see the section above. |
| 65 | + |
| 66 | +### Changed only the dist packaging (e.g., `apm-dist/`, log4j2.xml) |
| 67 | + |
| 68 | +```bash |
| 69 | +./mvnw -pl apm-dist -am -o clean flatten:flatten package -Pbackend,dist -DskipTests |
| 70 | +make docker.oap |
| 71 | +``` |
| 72 | + |
| 73 | +### Changed only Dockerfile / entrypoint (`docker/oap/*`) |
| 74 | + |
| 75 | +The tarball is untouched, so `make docker.oap` alone is correct: |
| 76 | + |
| 77 | +```bash |
| 78 | +make docker.oap |
| 79 | +``` |
| 80 | + |
| 81 | +## Verify the fix reached the image |
| 82 | + |
| 83 | +Docker's "image created" timestamp updates even when the content is stale (because `--no-cache` is used). **Don't trust the timestamp.** Verify the jar contents directly: |
| 84 | + |
| 85 | +```bash |
| 86 | +# Copy the jar out of the container |
| 87 | +docker cp <container-name>:/skywalking/oap-libs/<module>-<version>.jar /tmp/verify.jar |
| 88 | + |
| 89 | +# Extract the specific class |
| 90 | +cd /tmp && jar -xf verify.jar <path/to/YourClass.class> |
| 91 | + |
| 92 | +# Grep for a string your fix introduced (unique literal, method name, etc.) |
| 93 | +grep -oa "myFixSignature" <path/to/YourClass.class> |
| 94 | +``` |
| 95 | + |
| 96 | +If grep finds nothing, the image does not contain your fix — you forgot to rebuild the dist. Re-run with the correct chain. |
| 97 | + |
| 98 | +## Common pitfalls |
| 99 | + |
| 100 | +- **`make docker.oap` after `./mvnw package`**: The module jar is fresh in `oap-server/.../target/`, but the dist tarball at `dist/apache-skywalking-apm-bin.tar.gz` is untouched. Image uses the old jar. |
| 101 | +- **Cancelling a `make docker` mid-flight**: leaves the dist half-rebuilt. Re-run `make docker` or delete `dist/` first. |
| 102 | +- **Stale buildx cache**: not usually the issue (the Makefile passes `--no-cache`), but if you suspect it, run `docker buildx prune`. |
| 103 | +- **Trusting image timestamps**: `docker images` shows when the image was *tagged*, not when its content actually changed. A no-op rebuild still updates the timestamp. |
| 104 | +- **Interactive buildx hangs**: if `docker buildx build` sits silent for minutes with no stdout, kill it and retry. The buildx container driver occasionally stalls; a restart is faster than waiting. |
| 105 | + |
| 106 | +## The golden rule |
| 107 | + |
| 108 | +After any Java source edit that affects code shipped in `oap-libs/`, run one of: |
| 109 | + |
| 110 | +1. `make docker` (full, safest) |
| 111 | +2. `./mvnw -pl apm-dist -am -o package -Pbackend,dist -DskipTests -Dcheckstyle.skip=true && make docker.oap` (fast) |
| 112 | + |
| 113 | +Then **verify the fix is in the image's jar** before running e2e. Don't assume. |
0 commit comments