Android performance toolkit

September 11, 2026 · View on GitHub

This toolkit measures the optimized app rather than a debuggable Compose build. The Baseline Profile plugin derives two release-like variants from release:

devZapstoreBenchmarkRelease: minified, non-debuggable, profileable; used for Macrobenchmark.
devZapstoreNonMinifiedRelease: non-minified, non-debuggable, profileable; used to collect profiles.

Both use the Android debug key and the existing dev application ID (dev.ipf.whitenoise.android.dev). They cannot weaken or replace staging or production release signing. Direct adb install -r swaps preserve the dedicated fixture account's MDK SQLite state; the state-preserving runner below uses that path instead of AGP's connected-test install/teardown lifecycle.

The structure follows the Android team's Now in Android benchmark module and Macrobenchmark sample.

Host prerequisites

Install the Android SDK (including Platform Tools for adb), jq, and ripgrep (rg). The Baseline Profile verifier also needs either apkanalyzer from the Android SDK Command-Line Tools or unzip as a fallback. The scripts check these commands up front and report the missing dependency.

Local WNPerf device diagnostics

Debug, PR-preview, and staging builds expose a Performance logs switch under Settings → Developer → Diagnostics. It is off by default. Enabling it writes only the closed WNPerf timing schema to Logcat for up to 30 minutes; disabling it stops new events immediately. The session and its process-local counters are never persisted, so a process restart also disables ordinary device collection. The benchmark-selector build is the one exception: choosing that purpose-built local variant is the runner's explicit opt-in before its cold-start measurement.

To validate a slow journey on a staging device, enable the switch, reproduce one operation, then open App info → View logs and filter for WNPerf. Confirm the lines identify op, phase, layer, bounded timings/counts, and a closed result, and contain no account/group/message identifiers, npubs/pubkeys, URLs, filenames/paths, notification or message text, ciphertext, keys/tokens, or raw errors. Turn the switch off and repeat an action to confirm no new lines appear. White Noise controls future emission only; retention of lines already written to Logcat is controlled by Android/GrapheneOS.

MDK host timing integration

The 26 bridge stages use the published MarmotKit 0.9.20 recordHostTiming API. Product export now uses environment-specific Aptabase configuration and MDK's expanded, explicitly accepted Android consent scope. See Android usage and diagnostics for the variable names, first-launch/upgrade behavior, release gates, and pending operator validation.

Events (`app_` prefix)	Measured work
`text_send`, `text_reply`, `message_edit`, `message_react`	One native message command attempt
`media_upload`, `media_send`, `media_download`, `media_list`	Native transfer, publication or media listing
`timeline_read`, `message_search_page`	One timeline page/read, with search pages separated
`chat_list_read`, `chat_row_read`, `member_ids_read`	SQLite-backed chat/member projections
`profile_read`, `display_name_read`	Native profile/name reads
`account_list`, `unread_summary`, `catch_up`	Account listing, unread projection, catch-up
`group_create`, `invite_accept`, `group_roster`, `members_invite`, `members_remove`, `admin_promote`, `admin_demote`, `admin_self_demote`	Existing traced group operations

Each duration uses Android's monotonic elapsed clock, starts after IO dispatch and ends when the block returns or throws. It includes native suspension/queue wait, but excludes Android dispatcher admission, surrounding locks, UI decoding, layout and rendered frames. Cancellation is a failed host attempt, not proof that native publication failed. Retries and paginated reads are separate samples. Unknown trace names never enter the product registry or timing recorder.

Compare stage duration buckets and outcomes by app version/environment to locate slow call paths. Custom host events reach Aptabase only; they do not add OTLP series, raw-duration traces or exact percentiles. The upstream native queue, projection, acceptance and publication metrics provide the deeper breakdown. Do not add their percentiles to these overlapping host measurements. Existing Perfetto slices and the local WNPerf toggle remain independent of export consent.

Prepare a physical-device fixture

Use a dedicated API 34+ device with animations disabled and a stable power and thermal state. Emulator results are useful only as smoke tests; do not publish them as performance numbers.

Install the normal dev app: ./gradlew :app:installDevZapstoreDebug.
Sign in with a non-production test identity.
Create or receive a group with at least two members. Record its exact display name as GROUP_NAME.
For the one-shot invite benchmark, arrange a pending invitation and record its exact display name as INVITE_NAME. Re-create this fixture before each invite benchmark run because acceptance is intentionally irreversible.
Close any system overlays and keep the device awake and unlocked.

The journeys never clear package data. They use real UI actions and the real MDK store; no Android-side protocol cache or fake performance data is introduced. On a physical device, every connected-test command must pass -Pandroid.injected.androidTest.leaveApksInstalledAfterRun=true; otherwise AGP may uninstall the target during teardown and erase the local identity and message history. After profile collection, restore the normal dev APK in place without uninstalling or clearing data.

Run Macrobenchmarks

Isolated media component probe

For a diagnostic breakdown of native image downloads, encrypted Android cache reads/writes, and platform decoding, use MediaAttachmentLatencyProbe. This is an opt-in debug component probe, not a release qualification or UI first-frame benchmark. It uses the separate dev.ipf.whitenoise.android.medialatency package, never an existing dev, staging, or production store.

The probe creates a temporary identity and private group, uploads sixteen distinct generated approximately 64 KiB PNGs to the configured default media service, and verifies every download against the corresponding generated bytes. It reads no personal media or accounts. Only fixed phase names, sample counts, and aggregate durations are exported to instrumentation output. The test removes its temporary native store, cache, and cache key after closing the runtime; the remote encrypted fixture remains subject to the service's retention policy. Neither package is uninstalled.

The accompanying admission regression measures twenty memory-cache admissions and twenty platform-encrypted disk admissions without permitting a remote fetch. Each disk sample uses a fresh key to exclude memory-cache reuse; cleanup removes only entries created by the test. Admission timing is not composed-frame timing.

The encrypted-cache read/write and image-decode measurements are an independent test, so a failed network scenario cannot prevent their execution. To run just the offline local phases, append #measureLocalImagePhasesWithoutNetwork to the class selector below.

./gradlew --init-script scripts/media-latency.init.gradle \
  :app:assembleDevZapstoreDebug :app:assembleDevZapstoreDebugAndroidTest \
  -Pandroid.injected.build.abi=arm64-v8a
adb -s "$ANDROID_SERIAL" install -r -t \
  app/build/intermediates/apk/devZapstore/debug/app-dev-zapstore-arm64-v8a-debug.apk
adb -s "$ANDROID_SERIAL" install -r -t \
  app/build/intermediates/apk/androidTest/devZapstore/debug/app-dev-zapstore-debug-androidTest.apk
adb -s "$ANDROID_SERIAL" shell am instrument -w -r \
  -e class dev.ipf.whitenoise.android.media.MediaAttachmentLatencyProbe \
  -e allowMediaProbe true \
  dev.ipf.whitenoise.android.medialatency.test/androidx.test.runner.AndroidJUnitRunner

Verify both APKs' output-metadata.json application IDs before installation. The explicit init script changes only local diagnostic builds, and the test rejects any other target package. Do not use the general Macrobenchmark runner below for this probe: that runner intentionally replaces the existing dev app.

The network phase makes 20 sequential native calls for the first image with no Android plaintext cache, then 20 batches of the sixteen distinct references through the actual three-slot gate (340 native download calls total, roughly 22 MiB of image payload). The transport and server caches may already be warm; only the Android plaintext cache is bypassed. Each batch asserts the host concurrency bound and byte equality. Reports separate per-request latency including queue wait (320 samples) from whole-batch latency (20 samples). Local phases use 20 new encrypted-cache entries and the production decoder. Report first-sample effects through the maximum as well as nearest-rank p50/p95. The opt-in run has a ten-minute outer guard and may take several minutes.

After fixture upload, the live test snapshots the shipped native aggregate performance API before and after the download interval, including when a download fails. media_probe_native output contains only eleven fixed phase names, interval attempt/success/failure counts, duration sums, and numeric histogram buckets with an overflow count. It does not enable telemetry export or include unrelated snapshot fields. Native bucket counts are per-bucket, not cumulative; histogram bounds are not exact measured percentiles. Counter resets, saturation, or changed bucket bounds fail validation rather than producing a misleading interval report. A missing phase has zero observations, not a zero latency, and failed live runs remain failed even if the independent local tests pass. Do not discard failures or substitute the local timings for network timings.

This probe does not measure receipt-to-visible pixels in a real chat album, stalled-locator failover, per-request native phase attribution, internal HTTP concurrency, or a signed release candidate. Those require their own fixtures and representative-device runs; passing this diagnostic must not be treated as proof that all field latency tails are eliminated.

Deterministic Android media regressions

Run MediaDownloadHostRegressionTest for the actual app-state download and encrypted-cache path with a suspended synthetic native boundary, and MediaImageBubbleLatencyTest for the production Compose image bubble. The host tests assert distinct-request ownership, three-slot admission, explicit priority promotion, useful progress alongside a delayed native call, no host retry loop after native timeout/integrity failure, account isolation, and cache reuse after the caller leaves and returns. The Compose tests cover decoded thumbnail first-frame rendering with a frozen frame clock, cold encrypted-index hydration, and recomposition/navigation while a download is in flight. Cold cached image bubbles and album tiles now load even when automatic downloads are disabled. A cache-only read never falls through to the native download API: corrupt or evicted entries return to the explicit download affordance. The tests authenticate a deliberately damaged encrypted entry and assert zero native calls, and tracked screenshots cover standalone light/dark rendering and a loaded three-tile album. If policy grants network access during a failing cache-only read, materialization restarts with that new permission instead of leaving the image on a spinner. Separate bubble/tile regressions hold the real encrypted read across this policy change and assert exactly one native request followed by visible image content.

These tests do not reimplement native locator fallback. The pinned native artifact disables loopback blob endpoints, and its exported constructors do not expose the Rust-only opt-in. A mocked 10-second delay would not verify that implementation. The real 12-second fallback assertion therefore still needs a controlled public HTTPS service with a deliberately stalled first candidate and valid later candidate; do not weaken TLS or address validation to run it. Native HTTP concurrency and transport/integrity/crypto phase attribution remain separate from the Android ownership counters.

App journeys

Run startup plus the repeatable group-open journey without allowing AGP to remove the authenticated target package:

ANDROID_SERIAL=<device-serial> \
  scripts/run-performance-benchmarks.sh "$GROUP_NAME"

The script builds the normal dev and release-like APKs, replaces the dev app in place, invokes only the startup and group-open before/after methods, and pulls JSON plus Perfetto output into benchmark/build/outputs/manual/. Its exit trap restores the normal dev debug APK even when a benchmark fails. Both target and benchmark APKs are installed or updated in place; the runner never uninstalls a package, so authenticated app data remains intact on a personal physical device.

Before the controlled Macrobenchmark iterations, the runner also records the first cold launch after that in-place replacement. The explicitly selected release-like benchmark build emits privacy-safe WNPerf startup milestones for the system-splash handoff and the first authoritative local chat-list frame. The host runner requires a cold Activity launch, requires both milestones, and rejects a Compose handoff at or beyond two seconds. It writes the exact APK SHA-256, named device/API/build fingerprint, Activity launch timing, splash handoff timing, time to first app-owned Compose UI, and time to local Ready state to:

benchmark/build/outputs/manual/<UTC timestamp>/package-replacement-startup.json

The accompanying install, Activity launch, startup-log, and device-state files are retained beside the JSON so a published result is auditable. This journey measures an actual adb install -r package replacement and never clears app data; a normal dev debug APK is restored in place on every exit path.

For a focused rerun, set AndroidJUnitRunner's comma-separated class filter via BENCHMARK_CLASS_FILTER; the script still uses the same state-preserving path. The group title argument is optional when the selected method does not use the existing-group fixture. Every run retains instrumentation.log plus before/after device, battery, and thermal metadata beside its JSON and traces.

To capture the Android recovery resource baseline, use the guarded network journey on a supported physical Pixel:

ANDROID_SERIAL=<device-serial> \
  ALLOW_NETWORK_TOGGLE=true \
  BENCHMARK_CLASS_FILTER="dev.ipf.whitenoise.android.benchmark.NetworkRecoveryBenchmark#validatedNetworkRecoveryPower" \
  scripts/run-performance-benchmarks.sh

This explicit flag authorizes five controlled airplane-mode cycles. The runner captures the exact original airplane-mode state before replacing the APK and restores it from the exit trap on success or failure; the instrumented test has its own restoration fence as well. App data is preserved throughout because the runner continues to use only in-place APK replacement.

Each iteration starts from the authenticated chat list, settles online work, goes offline outside the measured trace, then measures a fixed 25-second offline-to-online recovery window. The result includes UI frame timing, peak target-process heap and anonymous RSS, system CPU/network/memory energy, the number and summed duration of bounded recovery attempts, native catch-up time, and any push-wake-lock duration. The accompanying Perfetto trace exposes target process scheduling for CPU-time analysis.

Treat the power categories as system-wide hardware energy, not app-exclusive attribution: the operating system and radio transition are part of the sample. Compare repeated runs on the same device, build, fixture, brightness, battery, network, and thermal state. Do not compare absolute values across devices, and do not set a fleet-wide threshold until representative baselines exist.

To measure group creation separately, use the state-preserving runner with an explicit mutation argument. This creates ten persistent MLS groups:

BENCHMARK_CLASS_FILTER="dev.ipf.whitenoise.android.benchmark.GroupFlowsBenchmark#createGroupConversationOpen" \
  CREATED_GROUP_PREFIX="Benchmark group <run-id>" \
  scripts/run-performance-benchmarks.sh

CREATED_GROUP_PREFIX is an explicit mutation guard: the creation benchmark is skipped unless it is supplied, because each of its ten measured iterations creates and syncs a persistent MLS group. Omit it when measuring only startup and the repeatable group-open journey.

To measure row tap to the first visible conversation transcript, run the warm re-open and cold-process variants against the same cached group fixture:

BENCHMARK_CLASS_FILTER="dev.ipf.whitenoise.android.benchmark.GroupFlowsBenchmark#openGroupConversationVisible,dev.ipf.whitenoise.android.benchmark.GroupFlowsBenchmark#openGroupConversationVisibleCold" \
  scripts/run-performance-benchmarks.sh "$GROUP_NAME"

The warm method opens and closes the conversation once during each unmeasured setup block. The cold method restarts the app process, waits for the local chat list, and then measures the first conversation open. Both end the measured firstTranscriptVisibleMs section when the transcript's anchored, non-empty Compose node first intersects the physical display. routeSettledMs covers the same tap through the frame after the 240 ms route tween completes, and frame timing spans that entire measured block. Warm setup waits for both the chat list's settled-route marker and disposal of the outgoing conversation controller, so the measured reopen cannot reuse the short exit-retention window. Report median, P90, and frame-overrun/jank metrics from the retained benchmark JSON; do not substitute debug-build timings.

Run the one-shot invite journey by filtering to its test method:

BENCHMARK_CLASS_FILTER="dev.ipf.whitenoise.android.benchmark.GroupFlowsBenchmark#acceptInviteConversationReady" \
  INVITE_NAME="$INVITE_NAME" \
  scripts/run-performance-benchmarks.sh

Run the two scroll journeys against the same cached fixture. Neither needs a group-name argument: the chat-list journey resumes to the list, and the conversation journey takes the group it opens from $GROUP_NAME.

BENCHMARK_CLASS_FILTER="dev.ipf.whitenoise.android.benchmark.ChatListScrollBenchmark#chatListScrollBaselineProfile" \
  scripts/run-performance-benchmarks.sh

Both scroll benchmarks report frame timing, a journeyDurationMs trace section, and peak process memory for the measured window: memoryHeapSizeKb, memoryRssAnonKb, and memoryGpuKb. Read the memory values as a budget rather than a target — decoded avatars, group images, and attachment buffers all grow while a long list scrolls, and a jump there with unchanged frame timing points at cache sizing rather than at rendering. Compare against the same fixture on the same device; the absolute values are not portable across devices.

Invite acceptance consumes its fixture, so collect a ten-sample handoff set by running this command once for each of ten distinct prepared invitations and aggregate their journeyDurationMs values. Never reuse a consumed invite.

StartupBenchmark reports timeToInitialDisplayMs and frame timing with no compilation and with the packaged Baseline Profile. GroupFlowsBenchmark reports journeyDurationMs, frame timing, and a Perfetto trace for group open → members visible, group creation → conversation ready, and invite acceptance → conversation ready.

StartupBenchmark also reports one trace section per bootstrap stage, so a regression names the stage that moved instead of only the total: client-construction, privacy-runtime-configuration, marmot-start, notification-platform-setup, notification-privacy-setup, account-refresh, account-activation, draft-reconciliation, and external-signer-registration. Each is the summed time inside that stage for the iteration, so a bootstrap that retried counts every attempt.

The underlying slices are named WhiteNoise.startup.<stage>; search the Perfetto slice table for WhiteNoise.startup. to see them beside the WhiteNoise.marmot.* bridge slices below. They are emitted only while a trace is active, and a section name is one of the fixed stage constants — never an account, group, or message identifier.

package-replacement-startup.json is a separate one-shot device journey. Its timeToFirstComposeUiMs is the conservative later value of Android's Activity launch time and the monotonic system-splash handoff; this prevents Application startup before the app trace exists from disappearing from the result. timeToReadyMs is the first locally authoritative chat-list frame measured by the process-local app_start trace; pre-AppState launch work remains represented by the separate Activity timing. Do not substitute emulator output for the named physical-device evidence required by the startup issue.

The same traces include async WhiteNoise.marmot.* slices for the awaited MDK calls in create, invite, accept, member-roster refresh, and admin flows. In Perfetto, search the slice table for WhiteNoise.marmot. to separate bridge time from Compose and coroutine scheduling time. For an ad-hoc 30-second capture outside Macrobenchmark, start this command and perform one flow before it ends:

adb shell perfetto -o /data/misc/perfetto-traces/whitenoise-groups.perfetto-trace \
  -t 30s -a dev.ipf.whitenoise.android.dev sched freq idle am wm gfx view binder_driver
adb pull /data/misc/perfetto-traces/whitenoise-groups.perfetto-trace .

Trace section names contain only operation names, never account, group, member, message, or relay identifiers.

The state-preserving script copies results and .perfetto-trace files under:

benchmark/build/outputs/manual/<UTC timestamp>/

./gradlew :benchmark:connectedCheck remains available for an emulator or CI device. Supplying groupName is required for authenticated group tests; group creation and invite acceptance also require their explicit arguments. Tests whose fixture or mutation argument is missing are reported as skipped. On a physical device, pass the leave-APKs-installed property shown above; prefer the state-preserving script for a local fixture because it also restores the debug APK in place.

Generate and package the Baseline Profile

Generate startup, chat-list, group-open, and member-roster rules on the prepared device. Only launch-to-chat-list rules enter the Startup Profile; the broader group and roster journey remains in the Baseline Profile so it cannot crowd startup code out of the primary DEX:

./gradlew :app:generateBaselineProfile \
  -Pandroid.injected.androidTest.leaveApksInstalledAfterRun=true \
  -Pandroid.testInstrumentationRunnerArguments.groupName="$GROUP_NAME"

The generated profile is merged into and committed from app/src/main/generated/baselineProfiles/. It is consumed by every supported release variant; generation does not run implicitly during ordinary release assembly. The leave-APKs-installed property prevents AGP teardown from removing the authenticated target. After collection, replace the profileable variant with the normal dev build in place—never uninstall first:

./gradlew :app:installDevZapstoreDebug

Build a release-like APK and verify both compiled profile assets:

./gradlew :app:assembleDevZapstoreBenchmarkRelease
bash scripts/verify-baseline-profile.sh \
  app/build/outputs/apk/devZapstore/benchmarkRelease/app-dev-zapstore-universal-benchmarkRelease.apk

The verifier uses apkanalyzer when available and falls back to the ZIP table. For a signed staging/production APK, pass that APK path to the same script.

Compose compiler reports

Generate the same optimized staging report that CI uploads:

./gradlew :app:compileStagingZapstoreReleaseKotlin \
  -Pwhitenoise.enableComposeCompilerReports=true

Outputs land in app/build/compose-metrics/ and app/build/compose-reports/. CI publishes them as the compose-compiler-reports artifact for 14 days.

PR measurement table

Use one physical device and unchanged fixture for both runs. Report medians and the benchmark JSON artifact, and link the relevant traces.

Journey	No compilation median	Baseline Profile median	Delta
Cold startup → initial display	ms	ms	%
Open group → members visible	ms	ms	%