I summarized how large language models learned from large text corpora, used Transformer self-attention to model context, generated responses by next-token prediction, and were refined with human feedback to improve safety and relevance.

The externship tasks were completed in Colab notebooks. I prepared and ran Python code to create data structures, clean and normalize text, parse JSON into DataFrames, and apply image preprocessing steps (grayscale, denoise, contrast, threshold) to improve OCR readiness.

The project processed a sample SDF with PyMuPDF, inspected word-level and block-level outputs, noted table structure loss and column misalignment, and recommended line/block aggregation, regex with proximity logic, and table-specific parsing for reliable date/vendor extraction.

The task analyzed three OCR engines on CoA scans. I ran each engine, collected raw outputs, mapped bounding boxes to fields like lot number, vendor, expiration date, and reported that all three extracted those fields with minor format differences (e.g., date formats).

During the RAG pipeline project I converted PDFs to embeddings with sentence-transformers/all-MiniLM-L6-v2, chunked text into 512-token windows with 50-token overlap, implemented hybrid retrieval (vector + BM25) with a reranker, and produced example QA outputs showing retrieved answers.

Faced with a sample SDF query about quality control, I ran three RAG configurations varying top_k and similarity threshold, logged retrieved chunks and short answers, and found the top_k=8 run returned the most context (best coverage) though it included some redundant text.