ML Notes
What I found interesting about DeepSeek-V4 architecture
A personal technical reading note on MoE scaling, long-context inference, KV-cache pressure, and architecture/hardware co-design.
Reading note
Technical Writing
Research updates, mechanism audits, and lab notes
Current Research
Paired-Acquisition Neural Factorization: scanner signal and tissue identity
Testing whether paired acquisitions of the same tissue can reduce scanner/acquisition signal in pathology embeddings while preserving tissue identity. SCORPION, cross-backbone checks, and baseline stress tests.
2025–2026
External Validation
External multi-scanner validation on canine SCC
Independent canine SCC dataset validates the scanner-suppression / tissue-preservation pattern: scanner-probe accuracy reduced from 0.7529 to 0.3614 while paired cosine improved from 0.6960 to 0.7300.
2026
Allocation Study
Pair-repeat allocation: unique pair diversity vs repeated exposure
Investigating how allocation of paired acquisitions across unique pairs versus repeated exposures affects factorization quality and tissue-identity preservation.
2026
Mechanism Audit
Baseline stress tests: scanner suppression is not enough
Linear scanner projection and PCA baselines demonstrate that simple dimensionality reduction does not match the scanner-suppression / tissue-preservation tradeoff achieved by Paired-Acquisition Neural Factorization.
2026
Mechanism Audit
Acquisition-branch audit: branch separation is measurable
Post-freeze audit confirming that the acquisition branch retains higher scanner recoverability while carrying much lower tissue-identity retrieval than the biological branch.
2026
Mechanism Audit
Pair-structure boundary test: biological pairing structure matters
True same-region paired acquisitions preserve tissue identity best. Looser or random pairings can still suppress scanner signal but do not preserve tissue identity as well.
2026
Earlier Research
Earlier: PANDA / TransnnMIL / AttentionMIL experiments
Early computational pathology research exploring public benchmark datasets, PCam, PANDA prostate grading, multiple-instance learning architectures, and pathology ML evaluation. Research-only learning and prototyping.
2025
Earlier Research
Earlier: simulated federated pathology stress testing
Simulated-federation PANDA studies of FedAvg under dominant-site label corruption and systematic ordinal threshold bias. Cross-site blending and dominance-aware switching explored as robustness mechanisms.
2025–2026
Retrospective
Research infrastructure lessons from an early medical-AI prototype
Earlier prototype work taught me about reliability, security, data flow, and ML infrastructure. It was not production medical AI, not HIPAA-compliant, not diagnostic software, and not hospital-ready.
2026
Security
Building an 18-node security research lab
Most people collect stamps or build model trains. I decided to build a full-scale security research lab with 18 nodes, 6 VLANs, and comprehensive attack infrastructure.
15 Jan 2025