# ARCHS4 Public Data Maintenance Guide This document explains the architecture for serving **ARCHS4** (and GEO) public RNA-seq data in the DIY Transcriptomics course, detailing how browser-based WebAssembly (WebR) handles large-scale datasets and how instructors can update the data as ARCHS4 releases new versions. --- ## 1. Rationale: Desktop R vs. In-Browser WebR | Feature | Desktop R Workflow | WebR Workspace (Browser) | | :--- | :--- | :--- | | **Data Source** | Full HDF5 file (`mouse_gene_v2.3.h5`, ~18–20 GB; `human_gene_v2.3.h5`, ~35–40 GB) | Compact, pre-extracted RDS/matrix (~500 KB) in virtual memory | | **Execution** | Reads directly from local hard drive via `rhdf5::h5read()` | Adapter intercepts `h5read()` calls and returns the targeted cohort | | **Memory Limit** | Bound by machine RAM | WebAssembly memory sandbox (~2–4 GB max per browser tab) | | **Bandwidth** | 20–40 GB one-time download | < 1 MB instant load across any device (laptop, tablet, phone) | | **Student Code** | Standard course script (`h5ls`, `h5read`, `which(%in%)`, `DGEList`, `prcomp`) | **Identical syntax**—zero code modifications required | --- ## 2. How WebR Serves ARCHS4 Data in Step 4 When students execute `Step4_publicData.R` in the workspace: 1. **`library(rhdf5)`:** WebR's package loader initializes the ARCHS4 virtual adapter. 2. **`h5ls("mouse_gene_v2.3.h5")`:** Returns the full structural schema of the ARCHS4 database ($932,405\text{ samples} \times 53,511\text{ genes}$). 3. **`all.samples.mouse <- h5read("mouse_gene_v2.3.h5", "meta/samples/geo_accession")`:** Returns the accession list with the 12 target study samples (`GSM2310941`–`GSM2310952`) at known unique index positions. 4. **`expression <- h5read("mouse_gene_v2.3.h5", "data/expression", index=list(my.sample.locations, ...))`:** Returns the $12 \times 53,511$ count matrix. 5. **Downstream Pipeline:** `DGEList`, `cpm()`, `calcNormFactors()`, `prcomp()`, and `ggplot` faceted small-multiples execute natively in WebAssembly. --- ## 3. Instructor Workflow: Updating as ARCHS4 Expands Whenever the Ma'ayan Lab releases a new ARCHS4 version (e.g. `mouse_gene_v2.4.h5` or newer human releases) or if you decide to change the featured public dataset in Step 4, follow this 3-step workflow: ### Step 1: Run the Extraction Script in Local R On your local machine (where you have the latest `.h5` file downloaded): ```R # ============================================================================== # Script: extract_archs4_cohort.R # Purpose: Extract targeted GEO/SRA series from new ARCHS4 release for WebR # ============================================================================== library(rhdf5) # 1. Path to your downloaded ARCHS4 HDF5 file archs4_h5_file <- "mouse_gene_v2.latest.h5" # 2. Define the sample GSM accessions for your study cohort (e.g., GSE86884) target_samples <- c( "GSM2310941", "GSM2310942", # WT_unstim "GSM2310943", "GSM2310944", # Ripk3_unstim "GSM2310945", "GSM2310946", # Ripk3Casp8_unstim "GSM2310947", "GSM2310948", # WT_LPS.6hr "GSM2310949", "GSM2310950", # Ripk3_LPS.6hr "GSM2310951", "GSM2310952" # Ripk3Casp8_LPS.6hr ) # 3. Read metadata from HDF5 all_accessions <- h5read(archs4_h5_file, "meta/samples/geo_accession") sample_locs <- which(all_accessions %in% target_samples) genes <- h5read(archs4_h5_file, "meta/genes/symbol") titles <- h5read(archs4_h5_file, "meta/samples/title")[sample_locs] sources <- h5read(archs4_h5_file, "meta/samples/source_name_ch1")[sample_locs] characteristics<- h5read(archs4_h5_file, "meta/samples/characteristics_ch1")[sample_locs] # 4. Extract raw expression matrix (samples x genes) raw_expr <- h5read(archs4_h5_file, "data/expression", index = list(sample_locs, seq_along(genes))) # 5. Format into a standard R matrix (genes as rows, samples as columns) raw_expr <- t(raw_expr) rownames(raw_expr) <- genes colnames(raw_expr) <- all_accessions[sample_locs] # 6. Package into a compact extraction list extracted_data <- list( counts = raw_expr, samples = all_accessions[sample_locs], sample_titles = titles, sources = sources, characteristics = characteristics, genes = genes, total_database_samples = length(all_accessions), total_database_genes = length(genes), archs4_version = "v2.latest", extracted_on = Sys.Date() ) # 7. Save to the website repository Data/files/ directory (~500 KB) saveRDS(extracted_data, "Data/files/archs4_extracted.rds") cat("Extraction complete! File size:", file.size("Data/files/archs4_extracted.rds") / 1024, "KB\n") ``` ### Step 2: Commit the File to GitHub ```bash git add Data/files/archs4_extracted.rds git commit -m "data(archs4): update extracted mouse macrophage cohort to latest ARCHS4 release" git push origin main ``` --- ## 4. Future Options: The `archs4r` R Package The Ma'ayan Lab also maintains the **`archs4r`** package on GitHub (`MaayanLab/archs4r`). * **How it works:** `archs4r` is a wrapper around `rhdf5` that simplifies syntax: ```R # With archs4r package: library(archs4r) metadata <- a4.meta.samples("mouse_gene_v2.latest.h5", mySamples) expression <- a4.data.samples("mouse_gene_v2.latest.h5", mySamples) ``` * **Desktop usage:** Desktop students still need to download the 18 GB `.h5` file, but their code is more concise. * **WebR support:** The WebR adapter can support `a4.data.samples()` and `a4.meta.samples()` alongside `h5read()` if you transition course lectures to the `archs4r` syntax.