You push a commit, the pipeline triggers, and then… nothing. The run shows up in Azure DevOps with a spinning grey clock and the words “Queued” or “This job is waiting on a runner to come online.” Five minutes pass. Ten. The build never starts. Nothing in your YAML changed, the code compiles locally, and yet the pipeline just sits there. This is, by a wide margin, the most common first-week problem people hit with Azure Pipelines — and the frustrating part is that it is almost never a bug in your pipeline. The pipeline is fine. What is missing is an agent: a machine that is free, capable, and allowed to run your job. Until one shows up, your run waits in line forever.
This guide is the playbook for that exact moment. A queued-forever job has a small, knowable set of causes — you have no parallelism granted on the organization, the agent pool has zero agents (or none online), your job’s demands match no agent, every agent is busy with someone else’s build, or your free parallel-jobs grant is still pending Microsoft’s manual approval. Each looks identical from the outside (a stuck run) but has a different fix, and the whole skill is telling them apart in under two minutes. You will learn to read the screens that give the game away — Project settings → Agent pools and Organization settings → Parallel jobs — confirm each cause with the precise portal path or az devops command, and apply the right fix.
By the end you will stop staring at a spinning clock and guessing. When a run hangs you will know whether to request parallel jobs, register a self-hosted agent, fix a typo in a pool: block, wait out a busy queue, or fill in the free-tier grant form Microsoft requires for new organizations — the difference between a thirty-second fix and a lost afternoon.
What problem this solves
Azure Pipelines hides a lot of machinery so you can write a few lines of YAML and get a CI/CD run. That abstraction is wonderful right up until the run won’t start, and then it becomes a black box: the UI says “Queued” with no error, no log, no red X — just a job that never begins. There’s no exception to read because nothing ran. The information you need is real and visible, but it lives on settings screens most beginners never open, and without knowing which screen maps to which cause you burn an hour refreshing the pipeline page hoping it will “unstick” itself.
What breaks without this knowledge: a developer rewrites a perfectly good pipeline; a lead files a support ticket and waits two days for a free-tier grant they could have requested in five minutes; someone re-queues a stuck run ten times (which never helps — the queue was never the problem). Meanwhile the real cause — zero parallel jobs on a new organization, an offline self-hosted agent, or a pool name that doesn’t exist — sits there, perfectly diagnosable, ignored.
Who hits this: essentially everyone, but hardest on brand-new organizations (zero Microsoft-hosted parallel jobs until they fill in a grant request), self-hosted agent users (a stopped service or powered-off machine), and demands users (a typo that matches no agent). The fix is almost never “rewrite the pipeline” — it’s “find which of the five things is missing and supply it.”
Learning objectives
By the end of this article you can:
- Explain what an agent, an agent pool, and a parallel job are, and why all three must line up before any pipeline run can start.
- Tell the difference between Microsoft-hosted and self-hosted agents, and know which one is queuing and why.
- Diagnose a stuck run as one of five causes — no parallelism granted, empty/offline pool, unmatched demands, all agents busy, or a pending free-tier grant — and confirm each with the exact portal path or
az devopscommand. - Request the free Microsoft-hosted parallelism grant (the form new organizations must submit) and buy or scale paid parallel jobs when you outgrow the free tier.
- Register a self-hosted agent on a Linux or Windows machine from scratch, confirm it shows Online, and bring it back when it goes offline.
- Read a job’s demands against an agent’s capabilities and fix a mismatch.
- Apply the right fix fast using a symptom→cause→confirm→fix playbook, and put guardrails in place so the queue never silently stalls again.
Prerequisites & where this fits
You should have an Azure DevOps organization and at least one project (free to create at dev.azure.com), and a basic pipeline — a azure-pipelines.yml in a repo that triggers on push. You should know how to open Project settings (the gear icon, bottom-left) and Organization settings, and have Project Collection Administrator or at least agent-pool administrator rights, because some fixes (registering an agent, requesting parallelism) need elevated permissions. Comfort running a shell command on a VM or your laptop helps for the self-hosted-agent parts. No prior troubleshooting experience is assumed — this is a Basic-level guide.
This sits at the very start of the CI/CD & DevOps track. It assumes only that you have written, or copied, a minimal pipeline — the mechanics of stages, jobs, and triggers are covered in Azure Pipelines Fundamentals: Stages, Jobs, Templates & Triggers, which is the natural article to read alongside this one. The broader idea of what a pipeline is lives in CI/CD Pipeline Explained and CI/CD Concepts Deep Dive: Pipelines, Triggers, Stages & Agents. Once your runs start reliably, secrets and variables come next — see Azure DevOps Variable Groups, Secret Files & Key Vault Link.
When a run is stuck, the answer lives in one of three places, and knowing which to open is half the battle: Organization settings → Parallel jobs (billing/grants — explains no-parallelism and pending-grant stalls), Project settings → Agent pools (the agents themselves — explains empty pools and offline/busy/disabled agents), and your repo’s azure-pipelines.yml (the pool: and demands: — explains wrong pool names and unmatched demands). The run page itself shows the symptom message that points you at one of the three.
Core concepts
Four mental models make every later diagnosis obvious. Pin these down and the stuck-job screen reads itself.
An agent is a machine that runs one job at a time. An agent is software on a computer (a Microsoft VM, your laptop, an on-prem server) that picks up a queued job, runs its steps, and reports back. One agent runs exactly one job at a time — so with one agent and two ready jobs, one runs and the other waits, not because anything is broken but because there’s no free worker. This is the whole topic in one line: a queued job is a job waiting for a free, capable, permitted agent.
An agent pool is the group of agents a job is allowed to use. Your pipeline says pool: <name> — a vmImage (Microsoft-hosted) or a self-hosted pool like MyLinuxPool. The job runs only on an agent in that pool. If the pool is empty, or every agent is offline or busy, the job queues. Point at a pool that doesn’t exist (a typo) and it queues forever.
A parallel job is your permission to run a job at all — and you start with very little. Separate from whether an agent exists, Azure DevOps meters how many jobs you may run at once, called parallel jobs (or parallelism); each running job consumes one slot. Microsoft-hosted and self-hosted parallelism are billed and granted separately. The catch that traps newcomers: a new organization gets zero Microsoft-hosted parallel jobs by default — you must submit a free-grant form that Microsoft approves manually (often 1–3 business days). Until it lands, every Microsoft-hosted run queues. It’s anti-abuse throttling, and it’s the number-one cause of “my very first pipeline won’t start.”
Demands and capabilities are how a job and an agent are matched. A self-hosted agent advertises capabilities (node, java, docker, OS, custom tags). A job can declare demands, and the scheduler only sends it to an agent whose capabilities satisfy every demand. Add a demand the agent lacks — or misspell Agent.OS — and no agent qualifies, so the job queues while agents sit idle. Microsoft-hosted agents match on vmImage instead, so demand mismatches are mostly a self-hosted concern.
The vocabulary in one table
Before the deep sections, here is every moving part side by side. The glossary at the end repeats these for lookup; this table is the mental model:
| Term | One-line definition | Where it lives | Why a job gets stuck on it |
|---|---|---|---|
| Agent | Software on a machine that runs one job at a time | A VM / your laptop / a server | None free → job waits |
| Agent pool | A named group of agents a job may use | Project/Org settings → Agent pools | Empty or all-offline pool |
| Microsoft-hosted agent | A fresh VM Microsoft spins up per job | Azure Pipelines pool (vmImage) |
No parallelism / pending grant |
| Self-hosted agent | An agent you install and run yourself | Your own machine | Offline, unregistered, busy |
| Parallel job (parallelism) | Permission to run N jobs at once | Org settings → Parallel jobs | Zero granted → nothing runs |
| Free grant | The form a new org submits for free parallelism | Grant request form | Pending → MS-hosted jobs queue |
| Demand | A capability a job requires | demands: in YAML |
No agent matches → queues |
| Capability | A trait an agent advertises | Agent → Capabilities tab | Missing → demand unmet |
Microsoft-hosted vs self-hosted: which one is stuck?
The first fork in any diagnosis is which kind of agent your job wants — the causes and fixes differ completely. Look at the pool: block:
pool: { vmImage: 'ubuntu-latest' }(orwindows-latest,macOS-latest) asks for a Microsoft-hosted agent — a clean VM Microsoft creates per job and discards. You install nothing; the usual stuck-cause is parallelism (none granted, or a pending grant).pool: 'MyPool'(a name you created) asks for a self-hosted agent — a machine you registered. The usual stuck-cause is the agent being offline, unregistered, or busy, or a demand that matches no agent.
Knowing your row removes half the guesswork:
| Aspect | Microsoft-hosted | Self-hosted |
|---|---|---|
| Who provides the machine | Microsoft (fresh VM per job) | You (your VM / server / laptop) |
| What you install | Nothing | The agent software, once |
| Where you pick it | pool: { vmImage: 'ubuntu-latest' } |
pool: 'YourPoolName' |
| Speed to start | Seconds (but waits on parallelism) | Instant if an agent is online |
| Free tier | 1 free parallel job (after grant), 1,800 min/mo | 1 free parallel job, unlimited minutes |
| Most common stuck-cause | No parallelism / pending free grant | Agent offline, unregistered, or busy |
| You maintain it? | No | Yes (patching, disk, the agent service) |
| Caching tools/SDKs between runs | No (clean each time) | Yes (everything persists) |
The queued run’s own agent assignment message also tells you which side you’re on (decoded in the error-message table later): “no hosted parallelism” points at Microsoft-hosted, “waiting for an agent in pool X” at self-hosted.
Cause 1 — No parallelism granted (the new-organization trap)
The cause that gets almost every newcomer. You create a fresh organization, write a clean pipeline, push — and the run queues with “No hosted parallelism has been purchased or granted.” Your YAML is fine. Microsoft grants zero Microsoft-hosted parallel jobs to new organizations by default (an anti-crypto-mining measure); you must explicitly request the free grant.
Confirm. Open Organization settings → Parallel jobs. Two counters appear: Microsoft-hosted and Self-hosted. If Microsoft-hosted shows 0 (and you haven’t bought any), that’s your answer — and the queued run’s message corroborates it. There is no az command to grant free parallelism (it’s a web form by design); the CLI is only useful to confirm the project itself is fine:
az devops login
az devops configure --defaults organization=https://dev.azure.com/yourorg
az pipelines list --project MyProject -o table # rules out a YAML/project problem
Fix. Submit the free Microsoft-hosted parallelism grant request (Microsoft’s form for exactly this; it asks for your org name and a short justification). Approval is manual, typically 1–3 business days. While you wait, unblock with a self-hosted agent (its free slot is usually available immediately) or buy a Microsoft-hosted parallel job (provisioned right away). The grant types and what each gets you:
| Parallelism type | How you get it | Cost | Lead time | What it unblocks |
|---|---|---|---|---|
| Free Microsoft-hosted (1 job) | Submit the free-grant form | Free | ~1–3 business days (manual) | First MS-hosted runs, 1,800 min/month |
| Paid Microsoft-hosted (per job) | Org settings → Parallel jobs → buy | Per parallel job / month | Immediate | More concurrent MS-hosted runs, unlimited minutes |
| Free self-hosted (1 job) | Register a self-hosted agent | Free | Immediate (after agent online) | Self-hosted runs with unlimited minutes |
| Paid self-hosted (per job) | Org settings → Parallel jobs → buy | Per parallel job / month | Immediate | More concurrent self-hosted runs |
The trap inside the trap: people see “1,800 free minutes” advertised and assume their org already has the free job active. It doesn’t — the minutes only apply once the grant is approved and the parallel-job count is at least 1. Counter at 0 means no usable minutes, regardless of the headline number.
Cause 2 — The agent pool has no agents (empty or all offline)
Your job targets a pool that contains zero agents, or every agent in it is offline — the scheduler has nowhere to send the job. This is the dominant cause for self-hosted pools: the machine is powered off, the agent service stopped, or an agent was never registered.
Confirm. Open Project settings → Agent pools → <your pool> → Agents. Each agent shows Online (green) or Offline (grey); an empty list means none was ever registered, all-grey means the machines/services are down. The CLI shows the same:
# List the agent pools and their IDs
az pipelines pool list -o table
# List agents in a specific pool by ID, with their online status
az pipelines agent list --pool-id 12 \
--query "[].{name:name, status:status, enabled:enabled}" -o table
A status of offline for every agent (or an empty list) is the smoking gun. If there’s no self-hosted pool at all and you meant Microsoft-hosted, you targeted the wrong pool — see Cause 3.
Fix. Bring an agent online, or add one. If the machine is just off, start it and the agent service:
# Linux: run interactively, or start the installed service
./run.sh # or: sudo ./svc.sh start && sudo ./svc.sh status
# Windows: the agent installs as a service "vstsagent.<org>.<pool>.<name>"
Start-Service "vstsagent.yourorg.YourPool.AGENT01"
If no agent was ever registered, register one (full walk-through in the lab below). The states an agent can be in, and what each means for your queued job:
| Agent status | What it means | Will it run your job? | What to do |
|---|---|---|---|
| Online + Enabled | Connected and available | Yes (if not busy) | Nothing — you’re good |
| Online + Disabled | Connected but paused | No | Re-enable it in the Agents tab |
| Offline | Not connected (machine/service down) | No | Start the machine + agent service |
| Busy (Online) | Currently running a job | Not until it frees up | Wait, or add another agent |
| Pool empty (no agents) | Nothing registered | No | Register a self-hosted agent |
Watch for the subtle one: an agent that is Online but Disabled never picks up work, yet looks “present.” Check the Enabled toggle, not just the online dot.
Cause 3 — Wrong pool name or a demand that matches nothing
Your YAML points at a pool that doesn’t exist, or asks for a capability no agent advertises. The scheduler can’t find a match and queues the job. Two flavours:
- Wrong pool name.
pool: 'LinuxBuilders'when the pool is actually calledLinux-Builders, or you used a self-hosted pool name while intending Microsoft-hosted. Pool names are case-sensitive and must match exactly. - Unmatched demand. You added
demands: [ docker, NODE_VERSION -equals 18 ]but no agent in the pool has those capabilities (typo, or the agent genuinely lacks them). Even one unmet demand disqualifies an agent.
Confirm. Check the pool the run actually targeted (the run’s agent/pool line names it), then list the agents’ capabilities and compare them against your demands:
# Capabilities an agent advertises (system + user-defined)
az pipelines agent show --pool-id 12 --agent-id 3 \
--query "{name:name, system:systemCapabilities, user:userCapabilities}" -o json
In the portal, Agent pools → <pool> → Agents → <agent> → Capabilities lists everything the agent offers — a demand that doesn’t appear there is your culprit. If the pool name is wrong, az pipelines pool list shows the real names.
Fix. Make the YAML match reality — fix the pool name, correct or remove the demand, or add the missing capability. The two halves of the match:
| Mismatch | What you see | How to confirm | Fix |
|---|---|---|---|
| Pool name typo / wrong pool | Job queues; pool in run looks unfamiliar | az pipelines pool list shows real names |
Correct pool: to the exact name |
| MS-hosted vs self-hosted confusion | Targeted a self-hosted pool by mistake | Run says “waiting for agent in pool X” | Use pool: { vmImage: 'ubuntu-latest' } |
| Demand the agent lacks | Idle agents, job still queues | Capabilities tab missing the demand | Remove demand, or install + tag the capability |
| Misspelled capability name | Same as above | Capability key differs from demand key | Match the exact key (case-sensitive) |
Retired vmImage |
Error/queue on a removed image | Image not in current hosted list | Move to a supported image (e.g. ubuntu-latest) |
To add a missing user capability, set it on the agent in the portal (Capabilities → add a name/value), or define an environment variable on the agent machine and restart the agent (it auto-publishes env vars as capabilities). A correct demand block:
pool:
name: 'Linux-Builders' # must match the pool name EXACTLY
demands:
- docker # the agent must advertise a 'docker' capability
- Agent.OS -equals Linux # built-in capability; note exact casing
Cause 4 — Every agent is busy (you’ve run out of slots)
Nothing is misconfigured — you simply have more ready jobs than free agents or parallel-job slots. On the free tier you get one parallel job, so a second concurrent run (a second pull request, or a multi-job pipeline) waits for the first to finish. The run is healthy; it’s just in line.
Confirm. Open Project settings → Agent pools → <pool> → Jobs (running/queued view), or Organization settings → Parallel jobs for consumption. One slot in use with another run waiting is your answer; the run’s message says it’s waiting for available parallelism, not for an agent to come online.
# Check overall pipeline runs and their state
az pipelines runs list --project MyProject --status inProgress -o table
Fix. Three options, in order of cost:
- Wait — an occasional collision clears the moment a slot frees; no change needed.
- Add a self-hosted agent — a second free self-hosted slot (or more online agents) runs two jobs at once at no extra cost.
- Buy more parallel jobs — Organization settings → Parallel jobs → raise the count. Provisioned immediately, billed per job per month.
How concurrency actually adds up, so you can size it:
| You have | Concurrent jobs possible | What a 2nd simultaneous run does | To increase |
|---|---|---|---|
| 1 free MS-hosted job | 1 | Queues until the first finishes | Buy MS-hosted jobs, or add self-hosted |
| 1 free self-hosted + 1 online agent | 1 | Queues (one agent, one job) | Add agents and/or buy self-hosted jobs |
| 1 self-hosted job + 3 online agents | Still 1 (slot-capped) | Queues — slots, not agents, cap you | Buy more self-hosted parallel jobs |
| 3 paid MS-hosted jobs | 3 | Runs immediately (slot available) | Buy more if you regularly hit 3 |
The non-obvious gotcha: parallel-job slots cap you, not the number of agents — ten online self-hosted agents with one purchased slot still run one job at a time while the rest idle. You need both to scale concurrency.
Cause 5 — The free-tier grant is still pending (or you hit the minutes cap)
A close cousin of Cause 1, worth its own callout because the symptom is “it worked, then stopped” rather than “it never worked.” Two sub-cases:
- Grant still pending. You submitted the free-parallelism form but Microsoft hasn’t approved it yet. Microsoft-hosted runs keep queuing until the grant lands — there is nothing to fix but the wait (use self-hosted in the meantime).
- Free minutes exhausted. Once granted, the free Microsoft-hosted job includes 1,800 minutes/month. Burn through them and further Microsoft-hosted runs are blocked until the monthly reset (or until you buy a paid parallel job, which removes the minute cap entirely).
Confirm. Organization settings → Parallel jobs shows whether the grant is active (count ≥ 1) and your consumption. A pending grant shows 0; an exhausted month shows the minutes used against the cap. Billing/usage details also surface there.
Fix. For a pending grant, wait or unblock with self-hosted. For exhausted minutes, wait for the monthly reset, switch heavy jobs to a self-hosted agent (unlimited minutes), or buy a paid Microsoft-hosted parallel job (no minute cap). Two free-tier numbers worth memorising: a free Microsoft-hosted job caps each run at 60 minutes and the org at 1,800 minutes/month; a free self-hosted job allows 360 minutes/run with unlimited monthly minutes. Buying a parallel job lifts those caps.
Architecture at a glance
Hold this mental model and every stuck-job screen reads itself. A pipeline run is not something your code does to itself — it is a scheduler matchmaking three independent resources, and “Queued” means at least one isn’t ready. Picture three gates a job must pass, in order:
- Gate one — permission: is a free parallel-job slot available? If parallelism is zero (new org, pending grant) or all slots are consumed (everything’s busy), the job stops here — no agent is even consulted, because you’re not allowed to start one.
- Gate two — a live agent in the pool: the named
poolmust hold at least one agent that is Online and Enabled. An empty pool, an offline machine, or a paused agent stalls the job here. - Gate three — a capability match: among the online agents, one must satisfy every demand (and, for Microsoft-hosted, the
vmImagemust be supported). A typo’d demand or missing capability blocks the job even with idle agents standing right there.
So the diagnostic flow is the same three questions every time: free slot? → online enabled agent? → demands satisfied? The first “no” is your cause, and the run’s status message usually whispers which gate (“no hosted parallelism” = gate one, “waiting for an agent in pool X” = gate two, idle agents yet still queued = gate three). Read the gate, not the spinner, and the Agent pools and Parallel jobs screens answer almost every stuck-job question in under two minutes.
Real-world scenario
Nimbus Retail, a four-person startup, had just moved their storefront repo into a brand-new Azure DevOps organization. On day one an engineer pushed the first commit, the azure-pipelines.yml (targeting vmImage: 'ubuntu-latest') triggered, and the run sat at “Queued” with the message “No hosted parallelism has been purchased or granted.” The team assumed the YAML was wrong and spent the morning rewriting steps, swapping ubuntu-latest for ubuntu-22.04, and recreating the pipeline — none of which helped, because none of it was the problem. The run never reached an agent; it was stuck at gate one, permission, with the Microsoft-hosted parallel-job counter sitting at 0.
A glance at Organization settings → Parallel jobs would have ended the morning in thirty seconds: Microsoft-hosted 0, self-hosted 0 — new organizations get no free Microsoft-hosted parallelism until the grant form is approved. The team did two things. They submitted the free-grant request (approved two business days later, lifting Microsoft-hosted to 1 with 1,800 monthly minutes), and — because they couldn’t wait two days for a demo — registered a self-hosted agent on a spare Linux VM. The free self-hosted slot activated immediately, the agent showed Online, and after pointing the pipeline at pool: { name: 'Nimbus-Linux' } the next run started in seconds.
A week later two engineers opened pull requests a minute apart and the second build queued behind the first. Not zero parallelism this time — both slots (the granted Microsoft-hosted one and the self-hosted one) were momentarily in use. A concurrency collision, confirmed on the Parallel jobs consumption view, and they decided an occasional 90-second wait wasn’t worth a third paid slot. The lesson Nimbus took is the one this article teaches: a stuck run is a resource problem, not a code problem, and the Parallel jobs and Agent pools screens name the missing resource faster than any amount of YAML editing. Total spend stayed at ₹0.
Advantages and disadvantages
Understanding why Azure DevOps meters runs this way — and the trade-offs of each agent type — helps you pick the right unblock rather than just reaching for the credit card.
| Approach | Advantages | Disadvantages |
|---|---|---|
| Microsoft-hosted agents | Zero maintenance; clean VM every run; no machine to patch; supported images kept current | Capped free minutes (1,800/mo); cold each time (no caching); new-org grant delay; queues on parallelism |
| Self-hosted agents | Unlimited minutes; instant start when online; caches tools/SDKs; full control of the machine | You patch, secure, and babysit it; goes offline if the machine/service stops; you own the security surface |
| Free tier (1 parallel job) | Genuinely free; fine for one engineer / one repo | One concurrent job only; second run always queues; manual grant for MS-hosted |
| Paid parallel jobs | Real concurrency; no minute cap; provisioned instantly | Monthly cost per job; easy to over-buy if the real issue was a busy queue, not capacity |
When each matters: a solo developer or learning project lives happily on the free tier — accept the occasional queue. A small team building several PRs a day wants a couple of self-hosted agents (free minutes, cheap VMs) or 2–3 paid Microsoft-hosted jobs (no machines to run). The mistake to avoid is buying parallel jobs to “fix” a stuck run that was actually a pending grant or an offline agent — capacity you didn’t need won’t start a job blocked at a different gate.
Hands-on lab
This lab takes a queuing pipeline and gets it running by registering a free self-hosted agent (which works immediately, no waiting on Microsoft). It’s free — you only need a Linux machine you can SSH into (a small Azure VM, WSL, or your laptop) and an Azure DevOps org with a project.
Step 1 — Confirm the stuck state. Run any pipeline (or push a commit). It queues. Open Organization settings → Parallel jobs and note the Microsoft-hosted and self-hosted counts. If both are 0, you’ve reproduced Cause 1.
Step 2 — Create a self-hosted agent pool. Project settings → Agent pools → Add pool → type Self-hosted → name it Lab-Linux. Or via CLI:
az devops configure --defaults organization=https://dev.azure.com/yourorg project=MyProject
az pipelines pool list -o table # note there's no Lab-Linux yet; create it in the portal
Step 3 — Create a Personal Access Token (PAT). User settings (top-right) → Personal access tokens → New Token → scope Agent Pools (Read & manage). Copy it — you’ll paste it during agent config. Treat it like a password.
Step 4 — Download and configure the agent on your Linux machine. From the Agent pools → Lab-Linux → New agent → Linux screen, copy the download command. Then:
# On the Linux machine
mkdir myagent && cd myagent
# (paste the download/tar command from the portal, e.g.:)
curl -O <agent-package-url-from-portal>
tar zxvf vsts-agent-linux-x64-*.tar.gz
# Configure it (interactive prompts: server URL, PAT, pool, agent name)
./config.sh
# Server URL: https://dev.azure.com/yourorg
# Authentication type: PAT
# Personal access token: <paste your PAT>
# Agent pool: Lab-Linux
# Agent name: lab-agent-01 (accept default work folder)
Step 5 — Run the agent. Start it interactively first so you can watch it connect:
./run.sh
# You should see: "Connected to the server" then "Listening for Jobs"
Leave that terminal open. In the portal, Agent pools → Lab-Linux → Agents now shows lab-agent-01 as Online — and that green dot means your free self-hosted parallel job is active.
Step 6 — Point the pipeline at your pool and run it. Edit azure-pipelines.yml:
pool:
name: 'Lab-Linux' # use your self-hosted pool instead of vmImage
steps:
- script: echo "Running on my own agent!"
displayName: 'Hello from self-hosted'
Commit. Because an online, enabled agent now exists in the pool, the run starts within seconds instead of queuing — watch the steps execute in the agent terminal too.
Step 7 — Verify and explore. Confirm via CLI, then toggle the agent Disabled in the portal and re-run: the job queues again (Cause 2’s disabled variant), and re-enabling clears it — the whole article in miniature.
az pipelines agent list --pool-id <Lab-Linux-id> \
--query "[].{name:name, status:status, enabled:enabled}" -o table # status: online
Step 8 — Teardown. Stop the agent (Ctrl-C) and unregister it so it doesn’t linger as a stale offline agent, then delete the pool and revoke the PAT (and the VM’s resource group if you created one):
./config.sh remove # paste your PAT again when prompted
Common mistakes & troubleshooting
This is the heart of the article — the playbook you keep open when a run won’t start. Scan the table for your symptom, confirm with the exact path/command, apply the fix. It spans the basic causes (new-org parallelism, offline agents) and the less-obvious ones (slot-vs-agent confusion, disabled agents, retired images, scoping).
| # | Symptom | Root cause | Confirm (exact path / command) | Fix |
|---|---|---|---|---|
| 1 | First-ever run queues; “No hosted parallelism” | New org has 0 Microsoft-hosted parallel jobs | Org settings → Parallel jobs (MS-hosted = 0) | Submit the free-grant form; or buy a job; or use self-hosted now |
| 2 | Run says “waiting for an agent in pool X” | Pool has no agents, or none online | Project settings → Agent pools → X → Agents (empty/grey) | Register or start a self-hosted agent (./run.sh / svc.sh start) |
| 3 | Agents are listed but all grey (Offline) | Machine off, or agent service stopped | Agents tab shows Offline; az pipelines agent list status=offline |
Start the VM and the agent service; check network to dev.azure.com |
| 4 | Idle online agents, yet the job still queues | A demands: capability matches no agent |
Agent → Capabilities tab vs job’s demands | Remove/fix the demand, or add the capability and restart the agent |
| 5 | Job queues against a pool that “looks wrong” | pool: name typo or wrong pool referenced |
az pipelines pool list (compare exact names) |
Correct pool: to the exact, case-sensitive name |
| 6 | Second concurrent run queues behind the first | Only 1 parallel job; both PRs build at once | Org settings → Parallel jobs (consumption shows 1/1) | Wait; add a self-hosted agent; or buy another parallel job |
| 7 | 10 online agents, still only 1 job runs | Parallel-job slots cap you, not agents | Parallel jobs count (self-hosted) is 1 | Buy more self-hosted parallel jobs to match agent count |
| 8 | Worked yesterday, queues today, “no parallelism” | Free 1,800 MS-hosted minutes exhausted | Org settings → Parallel jobs (minutes at cap) | Wait for monthly reset; move heavy jobs to self-hosted; or buy a job |
| 9 | Grant form submitted, runs still queue | Free grant still pending manual approval | Parallel jobs MS-hosted still 0; no approval email yet | Wait 1–3 business days; unblock with a self-hosted agent meanwhile |
| 10 | Agent present and Online but never picks up work | Agent is Disabled (paused) | Agents tab → Enabled toggle is off | Toggle the agent Enabled |
| 11 | YAML pipeline errors/queues on vmImage |
Named a retired/unsupported hosted image | Image not in current hosted-image list | Switch to a supported image (e.g. ubuntu-latest) |
| 12 | Self-hosted job queues; “pool not authorized for pipeline” | Pipeline lacks permission to use that pool | Pool → Security; or the run’s authorization prompt | Authorize the pipeline (Permit) on the pool’s Security |
| 13 | Deployment job queues forever | Environment/Deployment pool differs from build pool | The deployment job’s pool:; Environments → checks |
Point the deployment at a pool with an online agent |
| 14 | Agent goes offline minutes after registering | PAT expired/insufficient scope, or clock skew | Agent diag logs (_diag folder); PAT scope |
Recreate PAT with Agent Pools (Read & manage); re-run config.sh |
| 15 | Stuck after migrating org / renaming pool | Pipeline still references the old pool name | pool: value vs az pipelines pool list |
Update YAML to the new pool name; re-queue |
Per-symptom detail on the ones that trip people most
The new-org grant (rows 1, 9). The single most common cause and the most misdiagnosed: the run gives no code error, it just won’t start. The tell is the “no hosted parallelism” wording and a 0 on the Parallel jobs screen. Don’t touch your YAML — submit the free-grant form, and if you can’t wait, register a self-hosted agent (the free self-hosted slot activates the moment an agent reports Online). People lose days here rewriting perfectly good pipelines.
Slots vs agents (rows 6, 7). A parallel job is a permission slot, an agent is a worker, and you need both. Ten online agents with one purchased self-hosted parallel job still run one job at a time — the others idle while jobs queue. When concurrency is the problem, check the slot count on the Parallel jobs screen first, not just the agent list.
Disabled-but-online agents (row 10). An agent can be connected (green dot) yet Disabled, so it never accepts work — easy to miss because it “looks present.” Check the Enabled toggle on the Agents tab, not just the online dot; someone usually disabled it for maintenance and forgot to flip it back.
Offline agents & pool authorization (rows 3, 12, 14). An agent goes Offline when its machine is off, the agent service stopped, the network to dev.azure.com is blocked, or its PAT expired — check the _diag log folder for the reason, and recreate the PAT with Agent Pools (Read & manage) if it won’t reconnect. Separately, a new pipeline may need to be authorized to use a pool the first time (a “needs permission to access a resource” prompt with a Permit button, or the pool’s Security tab); until permitted, the job waits as if the pool were empty.
Error & status-message reference
The messages Azure DevOps actually shows on a stuck run, decoded:
| Message / status you see | What it really means | Most likely cause | First check |
|---|---|---|---|
| “No hosted parallelism has been purchased or granted” | 0 MS-hosted parallel jobs | New-org grant not approved (Cause 1/5) | Org settings → Parallel jobs |
| “…pool does not have any agents” / “could not find any agents” | The target pool is empty | No self-hosted agent registered (Cause 2) | Agent pools → pool → Agents |
| “Waiting for an available agent” / “waiting on a runner” | All matching agents busy or offline | Busy queue (Cause 4) or offline (Cause 2) | Agents status; Parallel jobs usage |
| “No agent found in pool X which satisfies the specified demands” | No agent meets the demands | Unmatched demand / typo (Cause 3) | Capabilities vs job demands |
| “This pipeline needs permission to access a resource” | Pipeline not authorized for the pool | Pool authorization (row 12) | Click Permit, or pool → Security |
| “You have exceeded the free pipeline minutes” | Free 1,800 MS-hosted minutes used up | Minutes exhausted (Cause 5) | Parallel jobs consumption |
| Agent shows “Offline” in the pool | Agent not connected to the service | Machine/service down, PAT, network | _diag logs; restart service |
| Stuck “Queued” with no message | Generic wait — slot or agent | A gate not satisfied | Walk the 3 gates in order |
Best practices
- Request the free grant the day you create the org. Submit the Microsoft-hosted parallelism form before your first deadline — it’s manual and takes 1–3 business days. Don’t let approval latency become an incident.
- Keep a self-hosted agent as a backup unblock. One cheap always-on VM with a registered agent means you’re never fully blocked by a pending grant or exhausted minutes — the free self-hosted slot costs nothing but the VM.
- Run self-hosted agents as a service, not interactively.
./run.shin an SSH session dies when the session closes; install viasvc.sh(Linux) or the Windows service so the agent survives reboots and logoffs. - Run at least two agents in a shared self-hosted pool, so one agent being busy or down doesn’t stall the team.
- Match parallel-job slots to your agent count. Buying agents without slots (or vice versa) wastes one of the two — scale them together for real concurrency.
- Name pools clearly and reference them exactly — case-sensitive, no near-duplicates (
Linuxvslinux-builders). A naming typo is a silent forever-queue. - Use
demandssparingly and verify they exist. Every demand can disqualify all agents; check the Capabilities tab before adding one to YAML. - Watch the Parallel jobs consumption view so you see minutes/slots running low before they hit zero, not after a run queues.
- Open the right screen first. Train the team to open Parallel jobs and Agent pools the instant a run queues — those two screens answer 90% of cases in two minutes.
Security notes
- Least-privilege PATs. The token an agent registers with needs only Agent Pools (Read & manage) — never a full-access PAT — with a short expiry. A leaked over-scoped PAT can register rogue agents or read your pipelines.
- Treat self-hosted agents as part of your attack surface. A self-hosted agent runs pipeline code on a machine you own, often with network and secret access. Keep it patched, isolated (a dedicated VM, not a shared admin box), and don’t run untrusted PRs on it without manual approval.
- Lock down pool permissions. Use Agent pools → Security to control which projects and pipelines may use a pool; an unrestricted self-hosted pool lets any pipeline in the org run code on your agent.
- Don’t echo secrets while debugging a stuck run. Resist adding
printenv/Get-ChildItem Env:steps that dump variables — secret variables can surface in logs. Use the agent’s_diaglogs and the portal status instead. - Prefer managed identity over long-lived PATs for pipeline-to-Azure auth where supported, and store secrets in a linked Key Vault rather than YAML — see Azure DevOps Variable Groups, Secret Files & Key Vault Link.
Cost & sizing
The point of understanding the causes is that the cheapest fix is often ₹0 — the stuck run was a missing grant or an offline agent, not a capacity shortfall. Spend only when concurrency is genuinely the bottleneck.
- Free tier covers solo and learning work. One free Microsoft-hosted job (1,800 min/month, after the grant) plus one free self-hosted job (unlimited minutes) is enough for one engineer on one repo. Accept the occasional queue.
- Self-hosted trades machine cost for unlimited minutes. A small always-on VM (a B2s, ~₹2,000–4,000/month) gives unlimited build minutes and caching — often cheaper than extra Microsoft-hosted slots if you build a lot. You pay whether it’s busy or idle, so right-size it.
- Paid parallel jobs are the no-machines path to concurrency, billed per month with no minute cap (self-hosted slots are cheaper — you bring the compute). Buy them only when you regularly see two-plus runs queuing.
- Don’t over-buy. The classic waste is buying parallel jobs to “fix” a stuck run that was actually a pending grant or a disabled agent — paying for slots while the real blocker is at a different gate.
A rough monthly picture for a small team, and what each option buys:
| Setup | Concurrency | Rough INR / month | Best for | Watch-out |
|---|---|---|---|---|
| Free MS-hosted (1) + free self-hosted (1) | 2 (1 of each type) | ₹0 | Solo dev / learning | 1,800 MS-hosted min cap; grant delay |
| Free tier + 1 self-hosted VM (B2s) | 2+, unlimited minutes | ~₹2,000–4,000 (VM) | Frequent builds, caching | You patch/secure the VM |
| 2× paid MS-hosted parallel jobs | 2 concurrent, no minute cap | Per-job monthly fee | No machines to run | Pay per slot regardless of use |
| 2–3 self-hosted agents + matching slots | 2–3 concurrent, unlimited min | VMs + per-slot fees | Team with steady throughput | Size slots = agents, or you waste one |
Interview & exam questions
1. A brand-new Azure DevOps organization’s first pipeline queues forever with “no hosted parallelism.” Why, and what’s the fix? New organizations are granted zero Microsoft-hosted parallel jobs by default (an anti-abuse measure). The pipeline is fine; it has no permission slot to start. Fix by submitting the free Microsoft-hosted parallelism grant request (manually approved in ~1–3 business days), or unblock immediately by registering a self-hosted agent or buying a paid parallel job.
2. What is the difference between a Microsoft-hosted and a self-hosted agent? A Microsoft-hosted agent is a fresh VM Microsoft provisions per job and discards after — zero maintenance, but capped free minutes and no caching. A self-hosted agent is a machine you install the agent on and maintain — unlimited minutes, caching, and instant start, but you patch and secure it and it goes offline if the machine/service stops.
3. A job sits queued while several agents in its pool show Online. What’s the most likely cause? A demand the job declares matches no agent’s capabilities (a typo or a capability the agents genuinely lack). The scheduler won’t dispatch to an agent that doesn’t satisfy every demand, so idle agents don’t help. Confirm by comparing the job’s demands: against the agent’s Capabilities tab; fix the demand or add the capability.
4. You have ten online self-hosted agents but only one job ever runs at a time. Why? Parallel-job slots cap concurrency, not the number of agents. With only one self-hosted parallel job purchased, only one job runs at once regardless of how many agents are online. Buy more self-hosted parallel jobs to match the agent count.
5. What’s the difference between an agent being Offline and being Disabled? Offline means the agent isn’t connected — its machine is off, the agent service stopped, the network is blocked, or its PAT failed. Disabled means the agent is connected (Online) but has been paused and won’t accept work. Offline → start the machine/service; Disabled → toggle it Enabled.
6. Your CI worked yesterday but today every Microsoft-hosted run queues with a parallelism message. What changed? You likely exhausted the free 1,800 Microsoft-hosted build minutes for the month. Further Microsoft-hosted runs are blocked until the monthly reset. Confirm on Organization settings → Parallel jobs (minutes consumed); fix by waiting for the reset, moving heavy jobs to a self-hosted agent (unlimited minutes), or buying a paid parallel job.
7. Which permission scope should the PAT used to register a self-hosted agent have, and why does it matter? Only Agent Pools (Read & manage). It’s least-privilege: the agent needs to register and listen for jobs, nothing more. An over-scoped or long-lived PAT is a security risk (it could register rogue agents or read pipelines) and an expired one is a top cause of agents silently going offline.
8. A pipeline targets a self-hosted pool and queues with “this pipeline needs permission to access a resource.” What’s happening? The pipeline isn’t authorized to use that agent pool yet. The first time a pipeline references a protected resource it must be permitted — click Permit on the prompt, or grant it under the pool’s Security. Until then the job waits as if no agent existed.
9. What’s the fastest way to confirm whether a stuck run is a parallelism problem or an agent problem? Open two screens: Organization settings → Parallel jobs (is a slot available?) and Project settings → Agent pools → <pool> → Agents (is an agent Online and Enabled?). A 0 or fully-consumed slot count points at parallelism; an empty/grey/disabled pool points at agents. The run’s own status message usually names the gate too.
10. When does it make financial sense to run self-hosted agents instead of buying Microsoft-hosted parallel jobs? When you build frequently enough to bump the 1,800-minute Microsoft-hosted cap, need caching of tools/SDKs between runs, or require specific software/hardware. A small always-on VM (~₹2,000–4,000/month) gives unlimited minutes and can be cheaper than several paid Microsoft-hosted slots — at the cost of you maintaining the machine.
These map to AZ-400 (DevOps Engineer Expert) — implement a build infrastructure, agent pools, parallel jobs, and self-hosted vs Microsoft-hosted agents — and touch the pipeline-fundamentals portion of AZ-204 (Developer Associate). A compact cert-mapping for revision:
| Question theme | Primary cert | Objective area |
|---|---|---|
| MS-hosted vs self-hosted; agent pools | AZ-400 | Design and implement a build infrastructure |
| Parallel jobs, free grant, minutes | AZ-400 | Manage pipeline capacity and cost |
| Demands vs capabilities matching | AZ-400 | Configure agents and pools |
| PAT scoping, pool security | AZ-400 / AZ-500 | Secure the pipeline and agents |
| Pipeline triggers and YAML basics | AZ-204 | Implement Azure DevOps pipelines |
Quick check
- Your very first pipeline in a new organization queues with “no hosted parallelism.” Is your YAML wrong? What’s the actual cause and the fix?
- Several agents in the pool are Online, but the job still queues. Name the most likely cause and where you’d confirm it.
- True or false: adding more online self-hosted agents always increases how many jobs run at once.
- An agent shows a green Online dot but never picks up work. What single setting do you check?
- You hit a parallelism message today on Microsoft-hosted runs that worked fine yesterday. Give two possible causes and one immediate unblock.
Answers
- No — your YAML is fine. New organizations get zero Microsoft-hosted parallel jobs by default. Confirm on Organization settings → Parallel jobs (count = 0). Fix by submitting the free-parallelism grant form (approved in 1–3 business days), or unblock now with a self-hosted agent or a paid parallel job.
- A demand in the job matches no agent’s capabilities (typo, or the agents lack it). Confirm by comparing the job’s
demands:against the agent’s Capabilities tab (oraz pipelines agent show). Fix the demand or add the capability. - False. Parallel-job slots cap concurrency, not agent count. Ten online agents with one purchased self-hosted parallel job still run one job at a time. You must buy more parallel jobs to match the agents.
- The agent’s Enabled toggle — it’s Disabled (paused) even though it’s Online. Re-enable it on the Agents tab.
- Causes: (a) the free 1,800 Microsoft-hosted minutes for the month are exhausted, or (b) a billing/grant change reset your slot. Immediate unblock: move the job to a self-hosted agent (unlimited minutes), or buy a paid parallel job. Confirm on Organization settings → Parallel jobs.
Glossary
- Agent — software running on a machine (Microsoft VM, your server, or laptop) that picks up one queued job at a time, runs its steps, and reports results.
- Agent pool — a named group of agents that a pipeline job is allowed to run on; the job’s
pool:value selects it. - Microsoft-hosted agent — a clean VM Microsoft provisions for a single job and discards afterward; selected via
pool: { vmImage: '...' }. No maintenance, capped free minutes. - Self-hosted agent — an agent you install and run on your own machine; unlimited minutes and caching, but you maintain and secure it.
- Parallel job (parallelism) — your permission to run a given number of jobs at the same time; each running job consumes one slot. Microsoft-hosted and self-hosted parallelism are billed separately.
- Free grant — the request form a new organization submits to receive its free Microsoft-hosted parallel job; approved manually, typically in 1–3 business days.
- Demand — a capability a job requires (
demands:), used to match the job to a capable self-hosted agent. - Capability — a trait an agent advertises (installed tools, OS, custom tags). The scheduler matches demands against capabilities.
vmImage— the label naming which Microsoft-hosted OS image a job runs on (e.g.ubuntu-latest,windows-latest).- PAT (Personal Access Token) — a scoped token used to authenticate the agent during registration; for agents it needs only Agent Pools (Read & manage).
- Online / Offline — whether an agent is currently connected to the Azure DevOps service and able to receive jobs.
- Enabled / Disabled — whether an online agent is allowed to accept work; a Disabled agent stays idle even while connected.
- Pool authorization — the permission a pipeline needs before it may use a particular agent pool; granted via the Permit prompt or the pool’s Security settings.
- Build minutes — the metered Microsoft-hosted compute time (1,800/month free after the grant); self-hosted agents have unlimited minutes.
- Queue / Queued — the state of a run waiting for a free, capable, permitted agent to become available.
Next steps
You can now tell a stuck run’s five causes apart and fix each fast. Build outward:
- Next: Azure Pipelines Fundamentals: Stages, Jobs, Templates & Triggers — the YAML structure behind every run, now that yours actually start.
- Related: CI/CD Concepts Deep Dive: Pipelines, Triggers, Stages & Agents — the deeper model of how agents, stages, and triggers fit together.
- Related: CI/CD Pipeline Explained — the plain-English big picture of what CI/CD is doing for you.
- Related: Azure DevOps Variable Groups, Secret Files & Key Vault Link — wire secrets and config into your now-running pipelines safely.
- Related: Self-Hosted Runners: Autoscaling & Ephemeral on Kubernetes — when one self-hosted agent isn’t enough and you want a fleet that scales itself.