LOCKEDIN / FORWARD-DEPLOYED ENGINEERING 101
Understand it.
Then build it.
You can be an experienced engineer and still be new to agents. Start with what each thing is and how it helps a person do useful work. Explore 58 original introductions across software, agents, cloud, security and delivery. Start here without a model key; each topic connects to deeper instruction and practice.
A place to start.
Choose a short reading route, or look up a concept below.
- 101UnderstandPlain language, visual walkthrough, vocabulary
- 102Build with guidanceOriginal lessons, examples and knowledge checks
- 103Practise and explainRunnable labs, failure cases and handover
Plan a cloud deployment
Decide which model-serving responsibilities become yours when you operate the runtime.
See the 7-concept route
For engineers moving an application into a customer environment. This route introduces the concepts before provider setup and deeper labs.
- 01The environmentLocate the application, its data and the responsibilities shared with a cloud provider.
- 02NetworkingTrace how a request reaches a service before choosing public or private access.
- 03IdentityDistinguish reaching a service from being allowed to use its data or actions.
- 04Google CloudApply those boundaries to Google Cloud projects, regions and instances.
- 05AzureCompare how Azure subscriptions, resource groups and networks organize the same workload.
- 06AWSCompare AWS accounts, regions, Availability Zones and instances without assuming the services are interchangeable.
- 07Private inferenceYou are here
Bookmark this page to return to this concept. The number shows your place in the reading route; it is not an assessment result.
Model serving and capacity engineering
What does running a private LLM involve?
Private inference runs a model within an environment whose access and data paths you control. You need model weights, a serving runtime, compatible hardware, identity, monitoring and an operating team. NVIDIA Nemotron names a model family; NVIDIA NIM provides inference microservices. They are different layers.
Some enterprise constraints call for controlled inference placement, and an FDE must explain the operational tradeoff.
UNDERSTAND THE SYSTEM
Private inference moves model serving into your boundary
Private inference runs a model on infrastructure under a chosen organizational boundary. You must operate the model runtime, capacity, security and data paths; 'private' is a deployment property, not a guarantee.
Read the connections in plain English
- 01Authorized application → Access boundary
Authenticate the caller and constrain reachable services.
- 02Access boundary → Model-serving runtime
Queue, reject or limit work when capacity or policy requires it.
- 03Model-serving runtime → GPU + memory
The runtime schedules computation within available resources.
- 04Model-serving runtime → Controlled telemetry
Avoid accidentally exporting sensitive prompts through telemetry.
Check your understanding: Does putting a model on your own GPU make the whole application private?
No. Data may still leave through tools, telemetry, downloads or other APIs. Trace every data path and enforce the intended boundary.
CONNECT IT TO WHAT YOU KNOW
Start with something familiar.
Owning the engine means operating the engine room: capacity, maintenance, access and recovery become part of your responsibility.
Private placement does not automatically establish privacy or compliance. Check model licensing, data flows, telemetry, storage and network access. Hardware sizing needs the actual model architecture and workload.
Words worth knowing
- Weights
- The learned parameters of a model.
- KV cache
- Stored attention keys and values used during generation.
- Quantization
- Representing model values with reduced precision, with quality and compatibility tradeoffs.
Useful foundations
Open a definition and diagram here. Your lesson stays in place.
A trained statistical modelWhat is a large language model?
QUICK REFERENCE / 101
What is a large language model?
A large language model (LLM) is a trained neural network. Its learned numerical parameters and current input are used to calculate possible next tokens; a decoding process selects the continuation. A model can produce text, code or structured proposals. The application around it supplies context, decides what tools are available and controls real-world actions.
Choosing a model is one decision inside an application design, not the whole design.
Units of model input and outputWhat are tokens and token budgets?
QUICK REFERENCE / 101
What are tokens and token budgets?
Tokens are the units a model uses to represent content. A token can be a word, part of a word or another encoded element. Token counts affect context capacity, output limits, latency and often price. Characters, words and tokens are not interchangeable counts.
Unbounded loops and large tool responses can exhaust cost and capacity before useful work finishes.
Packaging and operating softwareWhat are containers and deployment?
QUICK REFERENCE / 101
What are containers and deployment?
An image is a reusable package containing an application's files and startup configuration. A container is one isolated instance created from that image; it can be running or stopped. Docker is a tool for building images and managing containers. Deployment adds the environment, identity, networking, storage and operating decisions needed to run the application.
A laptop demo must survive deployment, restarts and handover to a team that did not build it.
ONE EXAMPLE / FICTIONAL CASE
Follow the connections.
A team wants a maintenance assistant to use an approved model inside its environment with bounded simultaneous requests.
UNDERSTAND THE SYSTEM
Private inference moves model serving into your boundary
Private inference runs a model on infrastructure under a chosen organizational boundary. You must operate the model runtime, capacity, security and data paths; 'private' is a deployment property, not a guarantee.
See what moves and why.
Read the connections in plain English
- 01Authorized application → Access boundary
Authenticate the caller and constrain reachable services.
- 02Access boundary → Model-serving runtime
Queue, reject or limit work when capacity or policy requires it.
- 03Model-serving runtime → GPU + memory
The runtime schedules computation within available resources.
- 04Model-serving runtime → Controlled telemetry
Avoid accidentally exporting sensitive prompts through telemetry.
Check your understanding: Does putting a model on your own GPU make the whole application private?
No. Data may still leave through tools, telemetry, downloads or other APIs. Trace every data path and enforce the intended boundary.
Work through it, step by step.
Choose the model and runtime
Verify the license, supported hardware and intended task behavior before selecting a serving package.
What you have now: A compatible deployment candidate
Budget memory
Account for weights, runtime reserve and the key/value cache used for active sequences.
What you have now: A concurrency envelope
Measure and operate
Test quality, throughput, latency and recovery under representative load; retain a rollback path.
What you have now: Measured service limits
FROM RECOGNITION TO REASONING
Use the existing synthetic memory-budget explorer.
- 01
Change request concurrency and output allowance.
- 02
Explain why the same GPU capacity admits one setting and refuses another.
- 03
List the real measurements still needed before hardware selection.
A capacity argument that distinguishes estimates from benchmarks.
Compare your reasoning when you are ready
The explorer's fixed profile teaches reservation arithmetic. Real capacity depends on architecture, precision, runtime and workload, so its result cannot be advertised as measured Nemotron performance.
Keep notes in your own working copy. Moving through these cards does not record completion.
TURN THE IDEA INTO USEFUL WORK
You can now go deeper.
Private placement does not automatically establish privacy or compliance. Check model licensing, data flows, telemetry, storage and network access. Hardware sizing needs the actual model architecture and workload.
Academy lessons use your existing training access. Free guides stay open.