SKILL.md

September 18, 2026 · View on GitHub

What SceneView is

SceneView is a declarative 3D and AR SDK. One mental model across every platform:

Android — SceneView { … } (3D) and ARSceneView { … } (AR) composables. Filament renderer. Artifacts: io.github.sceneview:sceneview:4.37.0 and io.github.sceneview:arsceneview:4.37.0.
Apple (iOS / macOS / visionOS) — SceneView { } and ARSceneView { } SwiftUI views from the sceneview monorepo via Swift Package Manager (tag 4.37.0). RealityKit renderer.
Compose Multiplatform — SceneViewer(…), one composable from commonMain (io.github.sceneview:sceneview-compose, unreleased). Viewer subset only — model, orbit camera, key light, environment, tap. No AR, no custom materials, no post-processing. Android delegates to the Filament SceneView { } below; iOS needs a one-time renderer registration; Desktop draws a placeholder. Reach for it only when the ask is genuinely shared-source; for anything platform-specific, use the native API.
Web — sceneview-web@4 on npm (Filament.js + WebXR).
Flutter — flutter_sceneview plugin (PlatformView bridge; pub.dev name since #2735, directory flutter/sceneview_flutter/).
React Native — @sceneview-sdk/react-native@4 (Fabric bridge).
MCP — sceneview-mcp on npm — gives AI agents direct API access from chat.

Nodes are declared as composables / SwiftUI views inside the parent SceneView's trailing content. No imperative scene.addChild(node).

Always treat llms.txt in the repo root as the source of truth. It carries the full SceneView / ARSceneView signatures, every node type, every helper. URL: https://github.com/sceneview/sceneview/blob/main/llms.txt

Repo-side samples/android-demo/src/main/java/io/github/sceneview/demo/demos/ contains a working demo for every node type — when in doubt, read the demo, do NOT improvise an API.

When to use this skill

Trigger on any of:

"Build me a 3D viewer / AR app in Compose / SwiftUI."
"Load a .glb / .gltf / .usdz / .3mf model in Compose."
"Open the .3mf that ChatGPT / a slicer gave me, in 3D or in AR."
"Place a model on a detected AR plane / image / face."
"Render 3D on the web with Filament.js or WebXR."
"Bridge a 3D scene to Flutter or React Native."
"Show the same 3D model from shared commonMain code in a Compose Multiplatform app."
"Convert a 2.x / 3.x SceneView snippet to 4.x."

Skip for plain ARCore-SDK, Sceneform (deprecated), Unity, Unreal, or RealityKit projects that do NOT use the SceneViewSwift wrapper.

Compose Multiplatform: `SceneViewer` (viewer subset, unreleased)

sceneview-compose is the only way to write one 3D composable in commonMain. It is not a portable replacement for SceneView { } — it covers the model-viewer case and stops there, on purpose.

Choose it only when the ask is genuinely shared-source. If the app is Android-only, or needs AR, custom materials or post-processing, use the platform-native API instead; this façade will not grow to cover them.

// commonMain — compiles on Android, iOS and Desktop
SceneViewer(
    model = ModelSource.Asset("models/damaged_helmet.glb"),
    modifier = Modifier.fillMaxSize(),
    camera = rememberCameraState(distance = 4f),
    lighting = Lighting(intensity = 100_000f, castShadows = true),
    environment = EnvironmentSource.Default,
    onTap = { hit -> if (hit != null) println("hit at ${hit.position}") },
    onError = { error -> println("load failed: ${error.message}") },
)

Rules an agent must not get wrong:

ModelSource is a sealed interface, not a string: Asset(path), Bytes(byteArray) or Url(url). Url accepts http/https only and throws IllegalArgumentException on anything else, in commonMain, on every platform.
Bytes and Url must be self-contained (a GLB, or glTF with embedded buffers). Only Asset resolves sibling .bin/texture files. A .gltf with external resources passed as Bytes loads incomplete and silently.
A failed load has no pixels. The viewport keeps showing the environment, which is indistinguishable from a load still in progress. Pass onError — without it the only trace is the platform log under the SceneViewer tag. It fires both for an exception (missing asset, HTTP error) and for a malformed model Filament refuses to parse, where SceneViewerError.cause is null.
Platform status: Android renders (Filament). iOS renders (RealityKit) but the app must register SceneViewerBridge.factory once at launch — a KMP module cannot depend on a Swift Package. Desktop draws a visible placeholder; it is not wired yet. On an unwired platform this never throws and never shows an empty viewport.
usdz is Apple-only. Every platform accepts glTF and GLB.
Lighting maps approximately between Filament and RealityKit — a scene tuned on Android reads slightly differently on iOS. That is a property of the façade.

The minimal correct Android example

Verified against samples/android-demo/.../ModelViewerDemo.kt:

@Composable
fun ModelViewerDemo() {
    val engine = rememberEngine()
    val modelLoader = rememberModelLoader(engine)
    val environmentLoader = rememberEnvironmentLoader(engine)
    val modelInstance = rememberModelInstance(modelLoader, "models/helmet.glb")

    SceneView(
        modifier = Modifier.fillMaxSize(),
        engine = engine,
        modelLoader = modelLoader,
        environmentLoader = environmentLoader,
    ) {
        modelInstance?.let { instance ->
            ModelNode(
                modelInstance = instance,
                scaleToUnits = 0.3f,
            )
        }
    }
}

AR tap-to-place is the same shape with ARSceneView. Verified against samples/android-demo/.../ARPlacementDemo.kt:

@Composable
fun ARPlacementDemo() {
    val engine = rememberEngine()
    val modelLoader = rememberModelLoader(engine)
    val placedAnchors = remember { mutableStateListOf<Anchor>() }
    var latestFrame by remember { mutableStateOf<Frame?>(null) }

    ARSceneView(
        modifier = Modifier.fillMaxSize(),
        engine = engine,
        modelLoader = modelLoader,
        planeRenderer = true,
        sessionConfiguration = { _, config ->
            config.planeFindingMode = Config.PlaneFindingMode.HORIZONTAL_AND_VERTICAL
            config.lightEstimationMode = Config.LightEstimationMode.ENVIRONMENTAL_HDR
        },
        onSessionUpdated = { _, frame -> latestFrame = frame },
        onGestureListener = rememberOnGestureListener(
            onSingleTapConfirmed = { event, node ->
                if (node != null) return@rememberOnGestureListener
                val frame = latestFrame ?: return@rememberOnGestureListener
                val hit = frame.hitTest(event).firstOrNull {
                    it.trackable is Plane && (it.trackable as Plane).isPoseInPolygon(it.hitPose)
                }
                hit?.createAnchor()?.let { placedAnchors.add(it) }
            }
        ),
    ) {
        placedAnchors.forEach { anchor ->
            key(anchor) {
                AnchorNode(anchor = anchor) {
                    rememberModelInstance(modelLoader, "models/helmet.glb")?.let { instance ->
                        ModelNode(modelInstance = instance, scaleToUnits = 0.3f, isEditable = true)
                    }
                }
            }
        }
    }
}

Note: ARSceneView takes a sessionConfiguration: (Session, Config) -> Unit lambda — there is NO rememberARSession() helper, do NOT invent one. AnchorNode takes a com.google.ar.core.Anchor instance; create one via hit.createAnchor() after a hit-test on the latest Frame.

Critical rules (verified — do not break)

rememberModelInstance returns nullable. First recomposition returns null while loading. Always guard with ?.let { … } or ?:. Never !!.
Filament JNI is main-thread-only. The remember* helpers handle this. For imperative code use modelLoader.loadModelInstanceAsync (see llms.txt § Threading rules).

LightNode accepts both top-level params and apply = { … } for builder extras. The canonical form for intensity/color/direction is top-level (verified against LightingDemo.kt):

LightNode(
    type = LightManager.Type.POINT,
    intensity = 30_000f,
    direction = Direction(-x, -y, -z),
    position = Position(x, y, z),
    color = colorOf(r = 1.0f, g = 0.95f, b = 0.8f),
    apply = { falloff(6f) },   // only Filament-builder extras go here
)

type is com.google.android.filament.LightManager.Type (DIRECTIONAL, POINT, FOCUSED_SPOT, SPOT, SUN).

AR anchors come from ARCore. Build an AnchorNode with a real com.google.ar.core.Anchor from hit.createAnchor(). There are NO AnchorNode.image() / .face() / .plane() factory functions on Android in v4.2 — use AugmentedImageNode for tracked images and AugmentedFaceNode for face meshes (both in arsceneview package).
SceneView vs ARSceneView ship in different artifacts. Don't mix. 3D-only → io.github.sceneview:sceneview. AR → io.github.sceneview:arsceneview (it transitively includes sceneview).
.3mf needs no special handling — do not write any. A 3MF (what ChatGPT and every slicer emit for a printable model) goes through the same rememberModelInstance / loadModel* call as a GLB: ModelLoader detects it by its ZIP magic and converts it to GLB in memory (#3482). Never branch on the file extension or the MIME type to decide — Android reports neither reliably, and a shared .3mf routinely arrives as application/octet-stream with no name at all. If you genuinely need to identify a buffer, ThreeMfLoader.isThreeMf(bytes) in sceneview-core reads the bytes, on every platform. See references/recipes.md § Recipe: open a .3mf print. The conversion also scales the file's declared unit to metres and rotates the printer's Z-up to Y-up, so a 60 mm print is life-size and upright in AR — never add a scale factor of your own to "fix" it.
Only glTF/GLB, 3MF (Android) and USDZ (Apple) load. STL, PLY meshes and OBJ + MTL are open issues (sceneview/sceneview#3486, #3487, #3488), and 3MF is not wired into sceneview-web yet (#3491). Say the format is not supported and name the issue rather than inventing a loader for it.
Don't recompose-thrash the loaders. rememberEngine / rememberModelLoader / rememberMaterialLoader / rememberEnvironmentLoader belong at the top of the screen-level composable, NOT inside scroll lists or item composables.

Performance / hot paths

Never call a decomposing or allocating getter inside onFrame (or any 30–60 Hz loop). Set the whole node.transform = … once instead of writing position / quaternion / scale one at a time (one-at-a-time writes recompose the matrix and drift — issue #2187); use Mat4.copyColumnsInto(scratch) not Mat4.toColumnsFloatArray() for per-frame uniform uploads; prefer the TRS-tuple slerp(startPosition, startQuaternion, startScale, …) when you already hold the components. Reading node.worldPosition / worldQuaternion per frame is fine now — those are cached. Always load via rememberModelInstance / rememberNode (cached + main-thread). Full table: docs/docs/performance.md § Hot Paths & Allocation-Free APIs (audit umbrella #2263).

Toolchain pairing

This skill is most useful paired with the android-cli skill:

⛔ android run --apks=APK --activity=PKG/.MainActivity — do not use. Measured three times in this repo (#2796, #2854, #2990) — most recently on CLI 1.0.15498356, where it printed App loaded: and Debuggable: true, then rejected an activity the platform resolves fine, and installed nothing — leaving an older build on the device while a QA run measured it. Install with adb install -r APK and launch with adb shell am start -n PKG/.MainActivity, then confirm the device's lastUpdateTime actually moved (adb shell dumpsys package PKG). An install step that reports success is not evidence the binary on the device is yours.
android screen capture --annotate -o ui.png + android screen resolve --screenshot=ui.png --string="tap #N" — visual UI testing of a 3D scene.
android layout --pretty -o ui.json — Compose UI tree dump (the 3D viewport reports as a single AndroidView, so for in-3D tap targets you still need pointerInput / hit testing).
android docs search "compose canvas" — underlying Compose APIs.

Haptic feedback

io.github.sceneview.haptic.SceneViewHaptic wraps Android's Vibrator behind seven semantic presets plus low-level escape hatches. Get an instance with rememberHapticFeedback() inside a @Composable; it is a silent no-op (one Log.d, never throws) when the device has no vibrator or the consumer app omits the permission.

import io.github.sceneview.haptic.rememberHapticFeedback

@Composable
fun PlaceAnchorButton(onPlace: () -> Unit) {
    val haptic = rememberHapticFeedback()
    Button(onClick = { haptic.medium(); onPlace() }) { Text("Place") }
}

Presets: light() medium() heavy() success() warning() error() selection(). Escape hatches: continuous(intensity, durationMs) and pattern(events). cancel() stops an in-progress vibration — rememberHapticFeedback() calls it automatically on dispose.
The consumer app MUST opt in by adding <uses-permission android:name="android.permission.VIBRATE" /> to its manifest — the sceneview library does not auto-merge it.
The same API ships on iOS (SceneViewSwift.SceneViewHaptic) and Web (sceneview.haptic.*). See llms.txt § Haptic Feedback.

Resources

Cheat sheet — every public composable, node, and helper, with their actual signatures pulled from llms.txt.
Recipes — pointers to the working demo in samples/android-demo/ for each of the 13 canonical patterns. Read the demo file, copy from it. Do not improvise.
Migration to 4.x — see also docs/docs/migration.md for the full rename map.

Workflow guidance

When the user asks for a SceneView feature:

Confirm the platform. Android / iOS / Web / Flutter / RN — don't assume.
Pick the right entrypoint. SceneView { } for 3D-only, ARSceneView { } for AR. Mention the matching Gradle artifact.
Read the matching demo in samples/android-demo/.../demos/ before writing code. If you can't find one, fall back to llms.txt. Never invent an API.
Use the remember* helpers at the top of the composable. Never call raw constructors for Engine / ModelLoader.
Handle the null from rememberModelInstance with ?.let { … }.
For AR, remind the user about Manifest.permission.CAMERA and the com.google.ar.core <meta-data> entry.
If the user pastes 2.x / 3.x code, point them at docs/docs/migration.md and the local references/migration.md.