Faced a compulsory research task on LLMs, I explained transformer architecture and self-attention, described next-token training, and noted fine-tuning with human feedback. The submission produced a concise technical summary of how LLMs generate and refine language.

The project involved loading document datasets in Colab, cleaning and standardizing text with regex and libraries, processing JSON into flat tables with pandas, and applying histogram equalization plus Gaussian blur to scanned pages. The submission included Colab notebooks and an enhanced document…

I extracted text and bounding boxes from multi-page SDFs with PyMuPDF, inspected layout issues (tables, headers/footers), then parsed dates and fields using line/word-level extraction and regex; the deliverable documented accuracy, layout challenges, and suggested fixes.

The task compared three OCR engines on a scanned pharmaceutical PDF. I ran Tesseract, PaddleOCR, and EasyOCR, documented install friction and outputs, and concluded PaddleOCR gave the best layout-aware extraction while EasyOCR was the easiest to use.

The externship task required a full RAG pipeline. I selected an embedding model, implemented semantic chunking, combined vector and keyword retrieval into a hybrid retriever, and added a reranker. The deliverable included example queries and the AI responses produced by the pipeline.

I tested three RAG configurations (varying top_k, similarity threshold, and an LLM reranker) on an SDF query. I recorded retrieved chunks, ranked excerpts, generated answers, confidence ratings, and noted that all configs produced the same correct answer while retrieval mechanics varied.