# Salman Quazi — salmanq.com (Full)
# Director of Engineering, Microsoft CoreAI | Mountain View, CA
# Topics: LLM internals, software architecture, algorithms, engineering craft

## About
Personal technical blog by Salman Quazi. Posts explain how large language models
work under the hood, covering tokenization, attention mechanisms, constrained
decoding, function calling, tool use, and agentic architectures. Secondary topics
include software design patterns, .NET performance, and algorithms.

## Author
Name: Salman Quazi
Role: Director of Engineering, Microsoft CoreAI
Location: Mountain View, CA
github: https://github.com/splusq
x: https://x.com/splusq
linkedin: https://www.linkedin.com/in/salmanquazi/
About: https://www.salmanq.com/about/


## LLM Series

- https://www.salmanq.com/blog/microgpt-from-first-principles/
  A line-by-line walkthrough of Karpathy's 200-line GPT implementation, with ASCII diagrams covering embeddings, attention, backpropagation, and the Adam optimizer -- the same algorithm powering ChatGPT and Claude, just at toy scale.

- https://www.salmanq.com/blog/llm-special-tokens/
  Special tokens like <|im_start|> and <|im_end|> are the invisible structural grammar of chat LLMs -- atomic vocabulary entries that impose conversational structure on a next-token predictor.

- https://www.salmanq.com/blog/post-training-vs-in-context-learning/
  How large language models adapt to new tasks through two fundamentally different mechanisms: permanently updating weights via post-training, or steering behavior at inference time through in-context learning.

- https://www.salmanq.com/blog/induction-heads/
  How Anthropic researchers discovered a two-head attention circuit that explains why language models can learn from examples in their context window — and what it reveals about the structure of model intelligence.

- https://www.salmanq.com/blog/finetuning-model-weights/
  Fine-tuning modifies a model's weights to specialize it for a task, but how those weights change varies dramatically — from updating every parameter to injecting tiny low-rank matrices that can be hot-swapped at inference time.

- https://www.salmanq.com/blog/speculative-decoding/
  A small draft model proposes tokens cheaply; the large target model verifies them all in a single parallel pass. The output distribution is provably identical to running the target model alone — you get the speed of the small model without trading away quality.

- https://www.salmanq.com/blog/llm-constrained-sampling/
  LLM function calls produce valid JSON not because the model is perfectly reliable, but because a grammar engine masks invalid tokens at every sampling step.

- https://www.salmanq.com/blog/llm-built-in-tools/
  Built-in tools like code_execution are in-distribution -- the model was trained on their exact invocation patterns -- while custom function tools force the model to generalize from a schema it has never seen.

- https://www.salmanq.com/blog/llm-tool-namespaces/
  What stops you from naming a function tool 'code_interpreter'? Nothing -- and it still won't collide with the built-in one. The answer is namespace separation, from Harmony tokens to the Responses API.

- https://www.salmanq.com/blog/llm-tool-invocation-gap/
  Tool execution is moving inward -- from client-side function calls into provider-managed infrastructure -- creating a widening gap between what built-in and custom tools can do.

- https://www.salmanq.com/blog/side-effects/
  Functional programming didn't teach us to eliminate side-effects — it taught us to make them explicit. Now LLMs with tool access are forcing the same lesson, at a higher level.

- https://www.salmanq.com/blog/simplest-agent-loop/
  Strip away the frameworks and an AI agent is just a while loop — with the LLM deciding when to stop.

- https://www.salmanq.com/blog/llm-tool-call-hallucination/
  Tool call hallucination isn't random noise — it's the model activating trained patterns that don't match your actual tool list. Understanding the mechanism makes it debuggable.

- https://www.salmanq.com/blog/demystifying-ai-sdks/
  A clear three-tier mental model for the AI tooling landscape: API SDKs call models, multi-agent frameworks coordinate them, and coding agents do the engineering work autonomously.

- https://www.salmanq.com/blog/composing-mcp-tools-with-typescript/
  mcp-compose lets LLMs chain multiple MCP tools in a single TypeScript snippet executed in a sandboxed runtime, keeping intermediate data out of the context window and saving tokens.

- https://www.salmanq.com/blog/skills-vs-mcp/
  MCP tools land in the model's context as a flat, static schema at every ReACT iteration. Skills use a three-tier progressive disclosure strategy that keeps context lean until the capability is actually needed.

- https://www.salmanq.com/blog/agent-protocols-mcp-a2a/
  MCP handles how agents connect to tools; A2A handles how agents connect to each other. Skills and Toolbox fill the gaps above and below — and under the Linux Foundation, these layers are settling into a coherent stack.

- https://www.salmanq.com/blog/understanding-sandboxes/
  Containers were designed for packaging, not security isolation. A deep dive into how gVisor, nested hypervisors, and Firecracker each close the gap between container convenience and real multi-tenant security.

- https://www.salmanq.com/blog/bpe-how-models-see-text/
  Language models don't read characters or words — they read subword tokens produced by an algorithm that greedily merges the most frequent byte pairs. That representation is why models miscount letters, fumble arithmetic, and cost more in some languages than others.


## AI & Agents

- https://www.salmanq.com/blog/introduction-to-machine-learning-2/
  A pointer to Max Welling's free textbook for engineers who want a practical, accessible entry point into machine learning algorithms.

- https://www.salmanq.com/blog/alpha-beta-searching/
  Alpha-beta pruning optimizes minimax search by skipping subtrees that can't influence the final decision, saving massive computation in game trees.


## Architecture & Design

- https://www.salmanq.com/blog/oci-images-and-crane/
  OCI images are a content-addressable graph of blobs, not a single file — understanding the format explains why image pulls are fast, how multi-platform images work, and what crane can do that Docker can't.

- https://www.salmanq.com/blog/building-a-service-execution-pipeline/
  A walkthrough of building a chain-of-responsibility execution pipeline in C# where each link handles one concern -- logging, response, termination -- so features can be added or removed without code changes.

- https://www.salmanq.com/blog/composition-over-inheritance/
  Composing behavior by holding references to collaborating objects gives you more control than inheriting from a base class -- and usually produces more flexible designs.

- https://www.salmanq.com/blog/using-the-mvvm-pattern-on-web-applications-part-iii/
  Part III wires up the client: Knockout.js applies data bindings from the server-side ViewModel via WebSocket, completing a full MVVM loop between C# and HTML.

- https://www.salmanq.com/blog/using-the-mvvm-pattern-on-web-applications-part-ii/
  Part II builds the server side: a C# ViewModel that pushes property changes to the browser over WebSockets, plus a REST endpoint that lets the HTML page instantiate ViewModels by name.

- https://www.salmanq.com/blog/how-do-you-test-your-software/
  Unit tests do more than catch bugs -- they expose design flaws. If a class is hard to test in isolation, that is a signal to invert dependencies and program against abstractions.

- https://www.salmanq.com/blog/using-the-mvvm-pattern-on-web-applications-part-i/
  MVVM decouples UI logic from views through data binding and INotifyPropertyChanged. Part I explains the pattern's core concepts and why web apps need a binding mechanism to make it work.

- https://www.salmanq.com/blog/brief-introduction-to-prism/
  How to use PRISM's dynamic module catalog to decouple a Silverlight/WPF shell from its modules, enabling independent teams to onboard components without recompiling the host application.

- https://www.salmanq.com/blog/uncertaintly-principal-and-software-engineering/
  Heisenberg's Uncertainty Principle has a direct analog in software engineering -- the problem domain, the software itself, and human behavior all introduce irreducible uncertainty into every system we build.

- https://www.salmanq.com/blog/can-uis-produce-bugs/
  Confusing UI design can produce real bugs that are hard to catch because every user navigates differently -- especially in web apps where developers have less control over the user's path.

- https://www.salmanq.com/blog/object-databases-ordbms/
  When hierarchical data like nested GPS coordinates breaks relational normalization, object-relational databases let you define composable types that keep the programmer's model clean while the DBMS handles the mapping.


## Security & Infrastructure

- https://www.salmanq.com/blog/supply-chain-attacks/
  Supply chain attacks target the tools and dependencies you trust, not your code directly. Here's how to think about them across container images, package managers, and Debian's unique threat model.

- https://www.salmanq.com/blog/switched-to-hugo-from-wordpress/
  Migrating from WordPress to Hugo with GitHub Pages, GitHub Actions for CI/CD, and giscus for comments -- a fully free, static, and clean blogging stack.

- https://www.salmanq.com/blog/azure-service-monitoring/
  Windows Azure's built-in endpoint monitoring lets you test service connectivity from multiple continents and get response-time and uptime reports with almost zero setup.

- https://www.salmanq.com/blog/how-to-check-if-integrated-windows-authentication-is-available/
  A lightweight JavaScript trick to detect whether Integrated Windows Authentication is available, so you can seamlessly fall back to forms-based login for remote users.

- https://www.salmanq.com/blog/using-ssis-to-access-network-resources/
  SSIS packages run under SQL Server Agent's local system account by default, which can't reach network drives. Set up a domain proxy credential to give your packages the access they need.

- https://www.salmanq.com/blog/ctrlaltdel-twice/
  The hidden CTRL+ALT+DEL twice shortcut to reach the classic Windows XP login screen may be removed in future releases -- and that could lock you out of admin access.

- https://www.salmanq.com/blog/using-asp-net-to-authenticate-against-ad/
  Replace the clunky integrated Windows authentication dialog with a custom ASP.NET login form that validates users against Active Directory using DirectoryServices and LDAP.

- https://www.salmanq.com/blog/ftp-microsoft-isa-server-2004/
  FTP logins work fine through ISA Server 2004, but uploads fail because data transfers use dynamic ephemeral ports -- not just port 21. Fixing it requires opening a range of outbound TCP ports above 1000.

- https://www.salmanq.com/blog/search-for-pgp-key-given-a-keyserver/
  A one-liner gpg command to look up PGP keys from any keyserver, including LDAP-based servers and MIT's public key server.

- https://www.salmanq.com/blog/volume-backup/
  A practical incremental backup strategy: mysqldump on a cron schedule, Samba mounts for cross-platform access, and Windows 2000 Server Backup to tie it all together.

- https://www.salmanq.com/blog/random-numbers/
  System clocks and mouse movements are weak entropy sources for large random number sets. Random.org solves this by sampling atmospheric noise from an unoccupied radio frequency and exposing the results over a SOAP API.

- https://www.salmanq.com/blog/verifying-your-site-is-running/
  Service monitors like WhatsUp Gold confirm IIS is running, but that doesn't mean your site actually works. A PHP script that fetches pages, checks for expected content, and sends SMS alerts when something is wrong.


## Algorithms

- https://www.salmanq.com/blog/reverse-a-linked-list-recursively/
  Walking through the trick of reversing a singly linked list with a single-parameter recursive function -- force the stack deep, then rewire pointers on the way back up.

- https://www.salmanq.com/blog/recursion-in-sql/
  Oracle's START WITH / CONNECT BY syntax lets you traverse parent-child hierarchies directly in SQL -- something SQL Server lacks, forcing you to fall back to recursive C# functions or stored procedures.

- https://www.salmanq.com/blog/recursive-fun-c/
  A recursive approach to checking if a number is the median of a list -- no sorting required, just partition and compare subset lengths.

- https://www.salmanq.com/blog/optimizing-regular-expressions/
  Nesting Kleene star inside a group -- ([a-z]+)* -- causes catastrophic backtracking. Reducing it to ([a-z])* is equivalent and dramatically faster, but general-purpose regex engines can't always detect the simplification.

- https://www.salmanq.com/blog/torus-smart-array-part-ii/
  Using a torus (wrap-around) array to track vacuumed, unknown, and blocked cells for an intelligent Roomba -- no shifting needed regardless of where the robot starts.

- https://www.salmanq.com/blog/vacuum-cleaner/
  Roomba wanders randomly and hopes for coverage. We built a robot vacuum that tracks where it has been, cuts redundant passes, and cleans a room in a fraction of the time.

- https://www.salmanq.com/blog/solution-to-line-tracker/
  NQC implementation of a line-following algorithm for the RCX 2.0 that rides the edge between tape and floor color rather than naively toggling on and off the line.

- https://www.salmanq.com/blog/line-tracker/
  Instead of following the center of a black line, tracking one edge gives a LEGO Mindstorms robot a simpler and more reliable line-following algorithm.

- https://www.salmanq.com/blog/kolmogorov-complexity/
  Kolmogorov complexity measures an object's complexity as the length of the shortest program that describes it -- a simple definition with surprisingly deep implications.

- https://www.salmanq.com/blog/power-of-prolog/
  Three lines of GNU Prolog solve a simultaneous equation by declaring constraints over a finite domain -- no algorithm needed, just state the problem and let the solver find X and Y.

- https://www.salmanq.com/blog/constraint-logic-programming/
  PROLOG makes constraint satisfaction elegant -- define your constraints, and the language finds every solution through automatic backtracking.

- https://www.salmanq.com/blog/unification/
  PROLOG's unification looks like assignment but supports pattern matching across compound terms -- and bounds-checking prevents recursive self-references.

- https://www.salmanq.com/blog/bit-operations/
  XOR-ing a register with itself zeros it out in a single instruction, saving bytes over a MOV -- a classic low-level trick for when every byte counts.

- https://www.salmanq.com/blog/prolog/
  Logic programming flips the script: instead of writing an algorithm that computes something logical, you supply the logic directly and let Prolog figure out the computation. Every programmer should take at least one course in it.


## Other Topics

- https://www.salmanq.com/blog/rapamycin-discovery/
  A soil sample from one of the world's most remote islands led to one of biology's most important discoveries — a molecule that touches aging, cancer, immunity, and the very logic of how cells decide to grow.

- https://www.salmanq.com/blog/autophagy/
  In 2016, Yoshinori Ohsumi won the Nobel Prize for cracking open one of biology's most elegant survival mechanisms — the process by which your cells eat themselves to stay alive.

- https://www.salmanq.com/blog/causation-vs-correlation/
  Correlation is easy to find and seductive to believe. Understanding what actually causes what — and how to tell the difference — is one of the most useful thinking tools you can develop.

- https://www.salmanq.com/blog/series/llm/
  A curated reading path from LLM fundamentals to production agents — covering weights, attention, tool use, and the emerging agent protocol stack.

- https://www.salmanq.com/blog/improving-metabolic-function/
  Insulin sensitivity is the body's federal interest rate -- when it degrades, everything degrades. A deep dive into the biochemistry of diet, sleep, and exercise as levers for metabolic health.

- https://www.salmanq.com/blog/5-things-you-probably-didnt-know-about-net-websockets/
  Connection upgrades, simultaneous send/receive limits, teardown pitfalls, COM exceptions under load, and unobserved exceptions -- five gotchas in the .NET 4.5 WebSocket implementation that can silently break your app.

- https://www.salmanq.com/blog/self-awareness/
  Sebastian Junger's insight about why soldiers miss war applies to any profession: knowing what truly drives you -- not the grand narrative, but the kernel underneath -- is the key to lasting fulfillment.

- https://www.salmanq.com/blog/net-and-node-js-performance-comparison/
  An empirical benchmark pitting bare-metal .NET 4.5 HttpListener against Node.js cluster under 200 concurrent file-read-and-sort requests. Node.js wins on average, and .NET degrades further over time.

- https://www.salmanq.com/blog/improving-wordpress-site-speed-on-iis/
  A quick guide to client-side static file caching and URL compression in IIS to speed up WordPress for both first-time and returning visitors.

- https://www.salmanq.com/blog/ads/
  A reflection on Jeff Hammerbacher's famous quote about the best minds of a generation optimizing ad clicks -- and what that says about where talent is going.

- https://www.salmanq.com/blog/c-sharp-as-a-scripting-language/
  scriptcs uses the Roslyn compiler to let you run C# as a scripting language -- no project files, no solution, no classes, just code.

- https://www.salmanq.com/blog/ie-compatibility-and-w3c-validation/
  The X-UA-Compatible meta tag breaks W3C validation. Moving it from HTML into a custom HTTP header via web.config satisfies both IE and the validator.

- https://www.salmanq.com/blog/learning-to-program/
  Code.org offers a free, interactive way to learn programming with video walkthroughs and backing from industry leaders like Gates and Zuckerberg.

- https://www.salmanq.com/blog/asynchronous-anonymous-methods/
  .NET 4.5 introduced async anonymous methods, letting you write Task.Run(async () => await ...) for inline asynchronous lambdas.

- https://www.salmanq.com/blog/task-timeouts/
  A clean .NET 4.5 extension method that uses Task.WhenAny with Task.Delay to timeout any async operation -- no more timer callbacks and spaghetti code.

- https://www.salmanq.com/blog/better-presentations/
  pptPlex is a PowerPoint add-on that builds dynamic, zoomable presentations so your audience always sees the global context of your discussion.

- https://www.salmanq.com/blog/rendering-step-1-of-asp-net/
  Build a dynamic step indicator in ASP.NET that automatically adapts to the number of views in a MultiView or Wizard control.

- https://www.salmanq.com/blog/presentation-on-jquery/
  Slides and sample pages from a jQuery presentation given at a campus web publishers meeting, with more discussion on real-world jQuery usage to follow.

- https://www.salmanq.com/blog/automatic-documentation/
  GhostDoc auto-generates XML documentation comments in Visual Studio by parsing method names, parameters, and conventions -- turning a dreaded chore into a single right-click.

- https://www.salmanq.com/blog/script-task-readingreturning-data/
  How to pass data in and out of an SSIS Script Task using ReadOnlyVariables and ReadWriteVariables -- with the User:: namespace gotcha that trips everyone up.

- https://www.salmanq.com/blog/legacy-code/
  A one-line boolean assignment is clearer and more readable than the verbose if/else pattern found in so much legacy code.

- https://www.salmanq.com/blog/using-control-adapters/
  ASP.NET 2.0 Control Adapters let you override how server controls render HTML -- inject custom attributes like onfocus and onblur without touching every page.

- https://www.salmanq.com/blog/automatic-printing/
  A WScript-based approach to scheduled automatic printing on Windows by programmatically opening Internet Explorer, loading a page, and sending it to the printer without user interaction.

- https://www.salmanq.com/blog/four-part-naming-convention-in-sql-server/
  Getting "Server is not configured for DATA ACCESS" when using four-part names in SQL Server? One sp_serveroption call fixes it.

- https://www.salmanq.com/blog/finding-content-in-your-files/
  A one-liner Windows batch command that recursively searches all files of a given type for a specific string and dumps the results to a file.

- https://www.salmanq.com/blog/typed-toarray-from-arraylist/
  ArrayList.ToArray() returns an untyped object -- pass a System.Type argument and cast the result to get a strongly typed array in C#.

- https://www.salmanq.com/blog/partial-classes/
  Partial classes in .NET 2.0 let you split UI member declarations from business logic into separate files, but without discipline they can mask flat procedural code behind OOP keywords.

- https://www.salmanq.com/blog/xmlhttp/
  XMLHttp enables asynchronous client-server communication without page reloads -- turning stateless web pages into responsive, desktop-like applications.

- https://www.salmanq.com/blog/the-trailing-slash/
  Omitting the trailing slash on URLs triggers a 301 redirect, adding an extra round trip per request. At scale, that one character makes a measurable performance difference.

- https://www.salmanq.com/blog/old-bugs-are-harder-to-catch/
  A deadline-calculation bug hid for three years because no August had the 5th fall on a Friday until now. Calendar logic is a minefield of rare edge cases that only extremely detailed requirements can defuse.

- https://www.salmanq.com/blog/mysql-version-information/
  A quick one-liner: use SELECT Version() to check which MySQL version you're running.

- https://www.salmanq.com/blog/valuetype-boxing/
  Iterating over an ArrayList of structs with foreach silently copies each value type, so mutations never stick. Here's why boxing burns you and how to fix it.

- https://www.salmanq.com/blog/form-posts-and-cachecontrol/
  The dreaded "Page has Expired" warning on browser back-button presses is caused by cache settings. Switching CacheControl between no-cache and public gives you control over when browsers re-fetch vs. show cached pages.

- https://www.salmanq.com/blog/option-groups/
  The HTML optgroup tag lets you visually group items inside a select dropdown with non-selectable headers and automatic indentation -- a small feature most developers overlook.

- https://www.salmanq.com/blog/why-run-as-doesnt-work-on-explorer-exe/
  New explorer.exe instances hand off to the already-running desktop shell and exit, so "Run As" can never force a different user context. The workaround: launch iexplore.exe with runas instead.

- https://www.salmanq.com/blog/faster-table-rendering/
  Internet Explorer waits for a complete table before rendering, making large datasets feel slow. Using table-layout:fixed and periodic Response.Flush calls forces progressive display.

- https://www.salmanq.com/blog/re-useable-images/
  Using CSS absolute positioning to overlay text on a semi-transparent header image -- a simple technique for reusable, branded page headers without generating separate images for each title.

- https://www.salmanq.com/blog/logos-in-photoshop/
  A step-by-step walkthrough of recreating a Macintosh Panther-style logo in Photoshop using bevel, emboss, dodge, burn, and a glass reflection trick.

- https://www.salmanq.com/blog/roomba-maintaining-direction-part-iii/
  Using dual light sensors and a paper pinwheel on each motor to track wheel rotation and maintain straight-line or 90-degree-turn direction on an intelligent Roomba.

- https://www.salmanq.com/blog/outlooks-vcalender-using-asp/
  Dynamically generating .vcs files in ASP to attach Outlook calendar reminders to confirmation emails -- and why each enrollment needs a unique filename to avoid race conditions with the mail server.

- https://www.salmanq.com/blog/macintosh-like-light-css/
  A pure CSS header layout inspired by Macintosh design, using a three-image background where the center stretches to fit content.

- https://www.salmanq.com/blog/javascript-performance-part-ii/
  Running multiple JavaScript animations on a single timer thread by tracking discrete state per element, with a menu hover effect as a worked example.

- https://www.salmanq.com/blog/javascript-performance-part-i/
  Why setInterval outperforms setTimeout for JavaScript animations, and how to think about timer-based rendering in early 2000s browsers.

- https://www.salmanq.com/blog/using-xslt-to-view-graphical-poll-results/
  An XSLT stylesheet that counts poll responses, multiplies by an expansion factor, and sets div heights to render bar-chart-style graphical results -- no JavaScript needed.

- https://www.salmanq.com/blog/dynamic-contents/
  Using JavaScript to dynamically generate a page outline from heading tags, giving users a quick-navigation menu built straight from the document structure.

- https://www.salmanq.com/blog/date-format-using-xslt/
  XSLT has no built-in date formatting, so here is a concise template that converts numeric dates like 06/23/2004 into human-readable month-year strings.

- https://www.salmanq.com/blog/javascript-buffering-offscreen/
  Using JavaScript's offscreenBuffering to eliminate the flash of unstyled content on CSS-heavy pages by rendering the page in memory before display.

- https://www.salmanq.com/blog/sequential-preloading/
  A simple recursive JavaScript pattern that forces images to preload in strict sequence by chaining each load via the onload callback.

- https://www.salmanq.com/blog/speaking-about-events-technically/
  Why an image's onload event won't re-fire when you change its src -- it's a recursive-descent event tied to the body -- and how IE's readyState property provides a workaround.

- https://www.salmanq.com/blog/embedded-software/
  Embedded software is everywhere -- microwaves, cars, DVD players -- and tools like VeriLog now let computer scientists write for hardware devices the same way they write for PCs.

- https://www.salmanq.com/blog/unique-elements-in-xslt/
  A two-pass XSLT technique for deduplicating XML data: first sort the elements, then filter with preceding-sibling to keep only unique entries.

- https://www.salmanq.com/blog/css-word-wrap/
  A quick CSS fix to prevent horizontal overflow when using PRE or XMP tags inside a fixed-width div.

- https://www.salmanq.com/blog/hiding-elements-during-printing/
  Using CSS @media queries with separate screen and print stylesheets to selectively hide elements when users print a web page.

- https://www.salmanq.com/blog/accesskey/
  The HTML accesskey attribute lets you assign keyboard shortcuts to any element on a page, making web apps dramatically more navigable with a single attribute.

- https://www.salmanq.com/blog/sicp/
  MIT's Structure and Interpretation of Computer Programs is freely available online -- and its preface contains one of computing's best quotes about writing programs for people first.

- https://www.salmanq.com/blog/thinking-in-c/
  A free 1200+ page C++ book worth downloading -- Thinking in C++ covers the language comprehensively and is available at no cost.


## Site Structure
- Homepage: https://www.salmanq.com/
- Blog index: https://www.salmanq.com/blog/
- About: https://www.salmanq.com/about/
- Search: https://www.salmanq.com/search/
- LLM Series: https://www.salmanq.com/blog/series/llm/
- RSS feed: https://www.salmanq.com/index.xml
- Sitemap: https://www.salmanq.com/sitemap.xml
- llms.txt: https://www.salmanq.com/llms.txt