Stage 2 transforms the lab‑book foundation into a validated research platform. Every instrument pipeline is built on real data, not synthetic tests. The schema is designed for machine learning — and the infrastructure is now ready for Stage 3.
The project started with a messy desktop folder, years of Origin projects, and a growing set of manual spreadsheets. Stage 1 built the data‑capture layer — composition calculator, literature cross‑checking, and basic file import. Stage 2 adds the analytical layer: real, validated characterisation pipelines for every instrument in the lab, with results stored in a structured schema designed for machine learning.
NIST LaB6 validation, Kα2 stripping, R² quality, Scherrer crystallite size, d‑spacing. 170 peaks, mean R² ≈ 0.938.
Hc, Mr, BHmax, temperature coefficients α/β, entropy change ΔSm. Handles PPMS/MPMS3/ACMS, mass from filename.
Zeiss, JEOL, Tescan (16‑bit). Adaptive thresholding, phase fraction with operator judgment. 690 images imported.
The original design sketch included a full Docker Compose stack with MinIO, Redis, Jupyter, and a file‑watcher service. Real use showed that a leaner system — Postgres, a Python API layer, and a proper desktop interface — covered everything actually needed, without the operational overhead of running five services to enter one sample.
Knowing what to cut turned out to matter more than the original breadth of the plan.
alloy_lab ├── samples composition (JSONB), lineage, VEC/δ/ΔH_mix ├── synthesis method, temperature, atmosphere, duration ├── characterization one row per XRD/VSM/SEM measurement ├── properties extracted numeric results per measurement ├── literature_checks deduped matches from 3 external databases └── material_classes
| Pipeline | Validation | Key features |
|---|---|---|
| XRD | NIST LaB6, lattice param within 0.0001 Å | Background subtraction, Kα2 stripping, neighbour‑aware fitting, R², Scherrer size, d‑spacing |
| VSM | End‑to‑end on real production DB | Instrument detection (PPMS/MPMS3/ACMS), mass extraction, segmentation, Hc/Mr, α/β, ΔSm, BHmax |
| SEM | 16 real images (Zeiss/JEOL/Tescan) | Unified metadata, adaptive thresholding, auto footer crop, phase fraction with operator threshold |
| Table | Purpose |
|---|---|
| xrd_peaks | long format, one row per peak |
| xrd_features | wide format, ML‑ready, one row per sample |
| vsm_mh_details | Hc, Mr, flag, branch_found, BHmax |
| vsm_temperature_coefficients | α, β, R², n_points |
| compositions.composition_type | 'aimed' / 'measured' / 'reference' |
Three standalone tools, each launched from the Data Viewer with the file pre‑loaded. The operator adjusts parameters, confirms the result, and saves directly to the database. No manual transcription, no copy‑paste errors.
Live sensitivity controls, per‑peak accept/reject, Save to DB
Multi‑segment, per‑segment accept/reject, Save to DB
Live threshold slider, Otsu suggestion, Save to DB
The database now contains real, validated measurements from real samples, stored in a schema designed from the start to feed a machine learning model.
| Sample | XRD peaks | mean R² | VSM Hc (Oe) |
|---|---|---|---|
| RP1a | 57 | 0.94 | 198.6 |
| RP2a | 58 | 0.93 | — |
| RP3a | 55 | 0.94 | — |
| La103Fe115Mn01Si14 | — | — | — |
3 September 2026 — the database was migrated from a local Docker container on the M4 Mac to the home server (192.168.2.42), completing the infrastructure setup for Stage 3. All three machines — Mac, ML Desktop, and the server itself — now share a single canonical database.
.env updated on all machines (POSTGRES_HOST=192.168.2.42). Connections verified: Mac ✅ ML Desktop ✅ new sample added end‑to‑end. Full step‑by‑step record — every command, the dump/restore, the final architecture: Database Migration: Alloy Lab Moves to the Home Server →
The database now contains a complete, validated feature set ready for ML:
Proposed ML workflow: feature matrix (compositions + structural features) → Gaussian Process or Neural Network regression → property prediction + optimisation suggestions → new composition recommendations → back into calculator → new samples.
Stage 2 represents several months of careful, methodical engineering work: building instrument‑specific parsers validated against real lab data and certified reference standards, designing a database schema that serves both the lab‑book needs of today and the ML needs of tomorrow, and building interactive tools that enforce good scientific practice.
The project is now in a strong position for Stage 3. The hard, instrument‑specific work is done.
Here you can download the presentation of the project in pdf format :