Why signal source separation isn't intent: a safety-first framework and proxy evaluation for neuroad

BoburbekBoburbek Uzbekistan
edited November 10 in Research

Hey guys! I wanted to share an open-access research proposal and prototype evaluation I recently published on SSRN, exploring the gap between raw biosignal decoding and deterministic UI actions.Coming from audio stem separation (using Demucs to split mixed audio into stems) and building clinical software, I started looking at whether separating mixed EEG channels could predict interface choices. Anyone working with OpenBCI or BrainFlow knows the reality: isolating signal components is fundamentally different from inferring a user's intent or permission.

Here are the core sections from the paper:
Section 3.1 & 3.2 — ICA and Representation vs. Meaning: Why Independent Component Analysis strips ocular and muscle artifacts but cannot produce a clean "thought track," and why raw sampled voltage arrays contain zero built-in semantic meaning.

Section 4.5 — The "Change, Ask, No Action" Gate: Why passive cognitive state estimates should never automatically alter an interface. We propose a strict three-way gate where "doing nothing" is an intentional safety feature if signal quality drops or ambiguity is high.

Section 4.6 — Curriculum Learning & Negative Controls: Structuring a staged evaluation pipeline that tests decoders against raw signals, ICA-filtered data, cross-session drift, and pure random-noise baselines to prevent illusory correlations.

Section 7.6 — Proxy MVP Findings: Testing an open-source browser proxy (using typed input, MediaPipe blendshapes, and Gemini scoring for DOM/CSS changes). In live testing, the facial model predicted I was "calmed down" even when the resulting layout mutation made the page objectively worse, showing why passive state models need explicit keep/reject confirmation.

Full preprint: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7682552
Extension code: https://github.com/abdugafforovboburbekbiz-del/ai-bci-mutator

I'd love to hear how other researchers in the OpenBCI community design confirmation thresholds and negative baselines when mapping continuous EEG features to discrete software triggers!

Comments

  • For the proxy experiment, a useful baseline could offer the same candidate layout changes with an explicit keep/reject control, but no facial signal. Then compare unwanted changes, correction time and confirmation prompts per completed task. That would ask whether the extra state estimate helps the user, rather than whether its score improves after a mutation. Would you include an unchanged-layout condition too?

  • BoburbekBoburbek Uzbekistan

    Ah man, this is such a good reality check.
    You're right about the permission gap, in the paper's theory I drew out the Change/Ask/No Action gate, but in the actual MVP code I took the lazy shortcut and only wired auto-changes without building the interactive "Ask" UI yet. Testing layout proposals with purely manual keep/reject controls would definitely show whether the passive camera/EEG signal actually helps, or if it's just adding useless noise.
    And yes, 100% on adding a completely unchanged, static layout as a baseline. If an adaptive UI doesn't actually beat the default interface in raw task time, the whole thing is just a gimmick.
    Really appreciate the honest breakdown, definitely adding both of these to the next test run!

  • One wrinkle: I wouldn't make raw task time the sole pass/fail test. A layout could be slightly slower but prevent wrong actions or reduce effort. I'd choose the main outcome before running the comparison, while keeping errors and correction time visible. Counterbalancing condition order would also help separate practice from adaptation. The manual keep/reject condition needs the same candidate changes as the camera condition; otherwise the proposals themselves become another difference between the tests.

  • BoburbekBoburbek Uzbekistan

    @bcidecoded
    Hmm that makes total sense, I am new to this field so I would love to get your perspective on how to set this up:

    Right now Gemini generates CSS on the fly, which means the layout changes vary every run. To keep conditions identical across the manual and sensor groups, should I just lock it down to a few pre-set layout profiles (like standard declutter presets)?

    Since I don't have a wet lab, what's the most honest way to measure "reduced effort"? Would tracking things like mis-clicks and undo frequency plus a raw NASA-TLX survey be enough to show real signal?

  • I'd freeze a small bank of candidate changes first. They can still be Gemini-generated: generate them once, inspect and save the exact CSS, then replay the same bank in the manual and camera-assisted conditions on the same page/task. Keep the unchanged-layout condition too. Use equivalent task sets and vary condition order so practice does not get mistaken for an adaptation benefit.

    Mis-clicks and undos tell you about interaction costs; NASA-TLX gives you reported workload, not a direct reading of cognitive effort. Collect it after each condition with the same instructions, state how you scored it and keep the individual ratings visible alongside task time, errors and correction time. That would make a useful usability pilot. It would not by itself show that the camera estimate, or a future EEG signal, measures effort accurately. I would treat sensor validation as a separate question.

  • BoburbekBoburbek Uzbekistan

    @bcidecoded
    Thank you, this is really helpful! I will implement the changes you suggested: a frozen bank of preset layout changes, an unchanged-layout baseline, and manual keep/reject controls.

    I have another question about making progress without BCI hardware. Could public benchmarks such as MOABB be a realistic starting point for testing the signal-processing side separately? For example, motor imagery or P300 target detection, not reading thoughts or choosing interface changes.

    Separately, I could simulate the decision layer: use fixed preset changes, synthetic or replayed feedback, and test whether the system should change, ask, or do nothing under different uncertainty and permission rules. I am not claiming this would read thoughts or replace real EEG data. It would be a way to build and test the decision layer before hardware is available.

    My main question is: what should I learn and build first so I am not starting from zero later, preprocessing, cross-subject evaluation, calibration, uncertainty, or the interface/decision layer? Also, is it realistic to start with CPU-friendly classical pipelines such as CSP + LDA on a lower-range NVIDIA RTX laptop?

  • Yes, MOABB is a realistic place to start. I'd choose one motor-imagery dataset and one classical pipeline before tackling all those topics at once. CSP + LDA is a sensible CPU starting point; you don't need CUDA training for that pipeline. Start with one participant while checking memory and runtime on your laptop, then expand.

    MOABB's Tutorial 1 uses BNCI2014_001, LeftRightImagery and CSP + LDA. First get from the raw recording to epochs and labels, keeping subject/session metadata. Run the tutorial's within-session evaluation and save the settings, versions and per-session scores. Its binary score is ROC-AUC, so don't read 0.8 as 80% accuracy.

    Next hold out a session, keeping learned preprocessing and CSP fitting inside the training split. MOABB's CrossSessionEvaluation leaves each session out in turn; that is not automatically a forward-in-time test. For a deployment-style check, explicitly train on the earlier session and test on the later one. Cross-subject transfer and limited-data calibration can follow once that baseline is clear.

    In parallel, your decision layer can accept deliberately simulated inputs: uncertainty rises, permission is missing, a signal disappears, a user rejects a change. Check the resulting action and log it. Keep that test separate from EEG decoding: classifying cued imagery would not establish that the person wants a layout change. I'd aim first for a reproducible offline baseline and a small set of decision-rule tests, rather than a combined 'brain-controlled' demo.

  • BoburbekBoburbek Uzbekistan

    @bcidecoded
    Thank you very much, this is exact guidance I needed. I’ll correct the methodology and add these points to Phase 2 of my research!

Sign In or Register to comment.