# DIY Transcriptomics — Firebase Backend & Data Schema Guide A technical manual covering the Firebase cloud infrastructure, Cloud Firestore collections, data models, Firebase Storage structure, and security rules. --- ## 1. Firebase Services Overview The DIY Transcriptomics platform operates as a lightweight, serverless Learning Management System (LMS) powered by Firebase: ```mermaid flowchart TD Client["Browser Client
(Jekyll + WebR + JS)"] --> Auth["Firebase Authentication
(Google, GitHub, Email)"] Client --> Firestore["Cloud Firestore
(NoSQL Documents + Vectors)"] Client --> Storage["Firebase Storage
(Submissions / Figures)"] Client --> Functions["Firebase Cloud Functions
(Node.js / Gemini / RAG)"] ``` * **Firebase Authentication:** User identity, email verification, and role-based permissions. * **Cloud Firestore:** Real-time NoSQL database holding student progress, vector embeddings, collaborative table states, and study manifests. * **Firebase Storage:** Secure bucket storage for student figure uploads, PDF assignments, and automated GCS backups. * **Firebase Cloud Functions:** Backend serverless endpoints handling real-time token streaming (SSE), vector chunking, and administrative batch jobs. --- ## 2. Cloud Firestore Collections & Data Models ### `/users/{uid}` Stores the core profile and authorization metadata for each registered learner: ```typescript interface UserProfile { uid: string; email: string; displayName: string; photoURL?: string; createdAt: Timestamp; lastLogin: Timestamp; enrollment: 'in-person' | 'virtual'; // Enforced by admin tags: string[]; // e.g. ['TA', 'Honors'] course_review_status: 'none' | 'requested' | 'certified'; certified_at?: Timestamp; } ``` #### Subcollections under `/users/{uid}`: * **`/users/{uid}/page_notes/{lectureId}`:** Auto-saved markdown notes written by the student inside the Lecture Companion sidebar. * **`/users/{uid}/progress/{lectureId}`:** Video playhead tracking (last timestamp, percent watched, completed flag). * **`/users/{uid}/lab_progress/{labId}`:** Lab task checklists, draft links, and submission timestamps. * **`/users/{uid}/webr_progress/{stepId}`:** Tracks completion of WebR curriculum steps. Stores `completed`, `completed_at`, `error_count`, and `last_used`. * **`/users/{uid}/my_studies/{studyId}`:** Cloud backup of the student's custom count analysis manifests: ```typescript interface StudyManifest { studyId: string; name: string; sampleCount: number; geneCount: number; factors: Record; customScriptDraft: string; // Autosaved R script isPublicCompendium: boolean; // GEO / ARCHS4 vs local upload createdAt: Timestamp; updatedAt: Timestamp; } ``` *(Note: Large raw count matrices remain strictly on client-side IndexedDB to ensure 100% student privacy and respect Firestore document size limits).* --- ### `/ai_activity_logs/{logId}` Comprehensive telemetry capturing interactions with Ask DIY and the Lab Critic for auditing and RLHF training: ```typescript interface AIActivityLog { userId: string; lectureId: string; question: string; answer: string; mode: 'standard' | 'socratic'; enrollment: 'in-person' | 'virtual'; rating?: 'thumbs_up' | 'thumbs_down'; instructor_override?: string; tokens?: { prompt: number; completion: number }; timestamp: Timestamp; // Specific to Report Studio RLHF: table_merge_training?: { tableNum: number; original_draft: string; final_report: string; contributors: string[]; peer_comments_count: number; }; } ``` --- ### `/course_knowledge_embeddings/{chunkId}` Vector database powering cross-curriculum Retrieval-Augmented Generation (RAG): ```typescript interface KnowledgeChunk { chunkId: string; moduleId: string; // e.g. 'lecture-04', 'step-2' sourceType: 'transcript' | 'script' | 'reading'; title: string; url: string; // Relative jump link text: string; // ~300 words of content embedding: Vector; // 768-dimensional vector (FieldValue.vector) createdAt: Timestamp; } ``` --- ### `/lab_sessions/{pageId}` Manages real-time collaborative state for table groups during lab exercises: ```typescript interface LabSession { pageId: string; // e.g. 'lab-week-05' tables: { [tableNum: number]: { coordinatorUid: string; // Active Table Coordinator draft: string; // Editable collaborative markdown original_draft: string; // Immutable AI-generated baseline report: string; // Finalized report submittedAt?: Timestamp; } } } ``` #### Subcollection `/lab_sessions/{pageId}/tables/{tableNum}/comments/{commentId}`: Stores inline peer annotations created in the Report Studio: * `commentId`: Unique identifier * `authorUid`: UID of the commenting student * `authorName`: Display name * `selectedText`: Quoted snippet highlighted by the author * `commentText`: Feedback text * `resolved`: Boolean flag * `createdAt`: Timestamp --- ### `/webr_activity_logs/{logId}` Lightweight, privacy-preserving runtime telemetry for in-browser R executions: ```typescript interface WebRActivityLog { userId: string; eventType: 'execution_error' | 'step_completion' | 'mydata_upload' | 'mydata_run'; stepId?: string; studyId?: string; errorMessage?: string; // R error message (sanitized) sampleCount?: number; // Aggregate count only (zero data logged) geneCount?: number; timestamp: Timestamp; } ``` --- ### `/contact_messages/{messageId}` Inquiries submitted through the public Contact Us form: * `name`: Sender name * `email`: Sender email * `category`: 'general' | 'technical' | 'scientific' | 'feedback' * `message`: Body text * `status`: 'new' | 'read' | 'archived' * `createdAt`: Timestamp --- ## 3. Firebase Storage Bucket Structure * **`/submissions/{userId}/{labId}/{filename}`:** Student uploads for assignment credit (PNG/PDF plots, Quarto reports). * Enforces a 5 MB file size limit. * Restricted so students can read/write only within their own `{userId}` directory. * **`/backups/training-data/{timestamp}.json`:** Automated JSON exports of `ai_activity_logs`. * **`/backups/user-data/{timestamp}.json`:** Automated JSON exports of user profiles and rosters. --- ## 4. Security Rules Architecture ### Firestore Rules (`firestore.rules`) * **Student Privacy:** Students can only read and write their own documents under `/users/{request.auth.uid}/`. * **Field-Level Protection:** Students cannot modify protected profile fields such as `enrollment`, `tags`, or `course_review_status`. * **Instructor Privileges:** Users verified against the instructor allow-list have global read access across submissions and analytics. * **Payload Sanitation:** WebR activity logs enforce strict `in ['execution_error', 'step_completion', 'mydata_upload', 'mydata_run']` event validation. ### Storage Rules (`storage.rules`) * Read/write restricted to authenticated users uploading into `/submissions/{request.auth.uid}/`. * File types restricted to standard scientific artifacts (`image/png`, `image/jpeg`, `application/pdf`). --- *Associated Guides:* [`docs/COURSE_MAINTENANCE_AND_CONTENT_GUIDE.md`](/docs/COURSE_MAINTENANCE_AND_CONTENT_GUIDE.html), [`docs/INSTRUCTOR_ADMIN_GUIDE.md`](/docs/INSTRUCTOR_ADMIN_GUIDE.html), [`docs/AI_ARCHITECTURE_AND_RLHF.md`](/docs/AI_ARCHITECTURE_AND_RLHF.html)