LOCKEDIN / FORWARD-DEPLOYED ENGINEERING 101
Understand it.
Then build it.
You can be an experienced engineer and still be new to agents. Start with what each thing is and how it helps a person do useful work. Explore 48 original introductions across software, agents, cloud, security and delivery. Start here without a model key; each topic connects to deeper instruction and practice.
- 101UnderstandPlain language, visual walkthrough, vocabulary
- 102Build with guidanceOriginal lessons, examples and knowledge checks
- 103Practise and explainRunnable labs, failure cases and handover
HBM / CAPACITY- 01Model weights
- 02Active request state
- 03Runtime + headroom
Capacity, bandwidth and numerical precision
Why does GPU memory matter so much?
GPU memory holds model parameters and the working data needed to run requests. Capacity tells you how much fits; bandwidth tells you how quickly data can move. A model that fits by itself may still run out of memory when several long requests arrive.
This is the bridge from a model choice to a credible capacity plan, admission limit and procurement request.
CONNECT IT TO WHAT YOU KNOW
Start with something familiar.
Think of a database buffer pool: stored data is only part of the budget. Active queries and execution workspaces need room too. More concurrent work can exhaust capacity without changing the database itself.
The weight estimate parameters × bytes per parameter is only a starting point. Request state, temporary tensors, runtime overhead and safety headroom also need capacity. Lower precision can reduce storage but needs quality evaluation.
Words worth knowing
- HBM
- High-bandwidth memory used by many data-center GPUs.
- KV cache
- Stored attention keys and values reused during token generation; actual state requirements depend on model architecture.
- Quantization
- Representing values with a smaller numerical format, with a possible accuracy trade-off.
ONE EXAMPLE / FICTIONAL CASE
Watch the work take shape.
In the synthetic lab, 8 billion parameters at 16 bits require 16 decimal GB for weights alone. Add the explicitly supplied request and runtime budgets before comparing with a device's capacity. These are arithmetic examples, not a Nemotron benchmark.
HBM / CAPACITY- 01Model weights
- 02Active request state
- 03Runtime + headroom
Estimate weights
Multiply parameter count by storage bits and divide by eight. Label decimal GB separately from binary GiB.
A weight-only estimateFROM RECOGNITION TO REASONING
Find a memory admission boundary.
- 01
Run the downloaded capacity lab with its synthetic defaults.
- 02
Increase concurrency until the estimate exceeds capacity.
- 03
Explain the missing real-world measurements and compare GB with GiB.
An allowed and refused estimate with assumptions attached.
Compare your reasoning when you are ready
The local arithmetic teaches a budget boundary. It cannot establish a GPU recommendation, real throughput or actual model memory behavior without measurements on the chosen configuration.
Keep notes in your own working copy. Moving through these cards does not record completion.
TURN THE IDEA INTO USEFUL WORK
You can now go deeper.
Fits in memory is a capacity question; fast enough is a separate measured question.
Academy lessons use your existing training access. Free guides stay open.