Vector Embeddings & Semantic Search in Bioinformatics

🧬 Cross-Lecture Semantic Vector Search (RAG)

How AI Assistants Retrieve Knowledge Across All Lectures & Readings via 768-D Vector Embeddings

STEP 1: INDEX CURRICULUM

Course Knowledge Ingestion

🎬 Video Transcripts

Lectures 1–11 with Timestamps & Subtitles

📄 Reading PDFs & Manuals

38 Papers in /Reading/files/ (Kallisto, Sleuth, FastQC)

💻 Master R Scripts

Steps 1–8 (Code, Functions, Pipelines)

✂️ Chunked & Vectorized
gemini-embedding-001 (768-D Vector Array)

STEP 2: SEMANTIC SEARCH

Vector Space Matching

Student on Lecture 1:

"How do I run tximport in R?"

📐 Cosine Nearest-Neighbor Search:

• Embeds question to 768-D vector

• Matches semantic angle in Firestore space

• Unlocks concepts across full curriculum

🎯 Top Semantic Match (0.89 Sim):
• Lecture 4: Differential Expression
• Step 2 Script + Soneson 2015 PDF

STEP 3: GENERATE & CITE

Grounded Answer + Links

⚡ Ask DIY (Gemini 2.5 Flash Response):

To import Kallisto transcript abundances into R, use tximport to summarize counts to the gene level:

txi <- tximport(files, type="kallisto", tx2gene=tx2gene)

Verified Citations:

🔗 [Lecture 4 - Differential Expression] (Step 2)

📄 [Soneson2015_tximport.pdf] (Section 2.1)

📐 Mathematical Foundations: Chunking, 768-D Space & Cosine Matching

How continuous curriculum text is segmented, projected into high-dimensional geometric clusters, and matched by angular proximity

STAGE 1: SLIDING WINDOW CHUNKING

Boundary Preservation

Long transcripts are partitioned into overlapping windows so biological thoughts are never split across boundaries:

Chunk 1: Words 1 → 300

"FastQC quality control, adapter trimming..."

50-word overlap ➔

Chunk 2: Words 250 → 550

"...adapter trimming, pseudoalignment with Kallisto..."

50-word overlap ➔

Chunk 3: Words 500 → 800

"...Kallisto index construction, tximport mapping..."

💡 Why Overlap Matters:

Prevents sentences explaining key steps (e.g. tximport parameters) from losing context at chunk edges.

STAGE 2: HIGH-DIMENSIONAL SPACE

768-D Semantic Space Clustering

Concepts with related biological meanings naturally cluster together in 768 dimensions.

STAGE 3: COSINE MATCHING

Angular Distance Metric

COSINE SIMILARITY FORMULA:

            cos(θ) = (A · B) / (||A|| × ||B||)
          

Range: [-1.0 to +1.0] (1.0 = Exact Match)

Query: "How do I run tximport?"

➔ Cluster B (L4 / Step 2): cos(θ) = 0.892 ✓

➔ Cluster A (L1 / FastQC): cos(θ) = 0.412

➔ Cluster C (L6 / PCA): cos(θ) = 0.185

⚡ IN-MEMORY & FIRESTORE ENGINE:

            Ranks all course chunks in <5 milliseconds to assemble the prompt.

🌐 Lecture Companion: Dynamic DOM Context Discovery

How the client-side controller scans the browser Document Object Model (DOM) to ground Ask DIY in the student's active video and section

1. BROWSER DOM (LECTURE PAGE)

Document Object Model Elements

<h1 class="project-title">

Ultra-fast read mapping with Kallisto

PAGE TITLE ➔

<h3 id="part-1">

Part 1 - Step-by-step walkthrough of FastQC

SECTION HEADER ➔

<iframe vimeo-id="410750113">

ACTIVE STREAM ➔

▶️

              00:14:22 / 00:32:00
            

<a href="/Reading/files/FastQC_manual.pdf">

📄 FastQC User Manual (PDF)

LECTURE COMPANION

🎓 Tutor Mode: Active

💬 Ask DIY

📝 Notes

🎬 Videos

📚 Resources

🎯 Active Context: Part 1 (FastQC Walkthrough) ⏱️ 00:14:22

What does the red line on this FastQC quality score plot mean?

🎓 Ask DIY Tutor (Grounded in Video @ 00:14:22):

At 00:14:22 in Part 1, the red curve traces the median Phred score across each base position. If it drops below Q20, what does that tell you about base call accuracy?

🔗 Source: [Part 1 Video Transcript (00:14:14-00:15:08)]

Ask a question about this lecture...

➤