How to Use Just-in-Time Analysis in Scaled Agile Environments

Just over 2 years ago, I wrote a post on the Benefits of Just-in-Time Analysis, exploring the benefits of backlog refinement 2-6 weeks ahead of development taking place, vs. Big Up Front Requirements (BRUF), sometimes 3-6 months or even a year in advance.

Just-in-Time Analysis works well for self-contained Scrum teams doing software development, but what about more complex programmes of work with dependencies on other business units, some of which may need 3 months' notice of work coming their way?

Scaled Agile Frameworks (SAFe) handles this with a Planning Interval (also formerly known as Programme Increment or PI) Planning - a timebox of 8-12 weeks, containing 4-5 development iterations and a planning iteration. The planning session (AKA "Big Room Planning") typically lasts 1-2 days, with all the teams and stakeholders in one "Big Room" to plan the next 8-12 weeks' work.

This is all very well but, in my opinion, it detracts from two of the most fundamental principles and benefits of Agile:

• Being able to easily adapt to changing requirements and / or priorities

• Valuing productivity over process.

Ability to Adapt to Change

If PI Planning is held every 8-12 weeks, either:

1. The PO has to have a fully stocked backlog of refined and estimated user stories 12-16 weeks in advance of development starting, in order to be prepared,

or

2. Stories are not prepared in advance and stakeholders instead call out their requirements and priorities for the next 2-3 months during the planning session,

or

3. You get a worst-of-all-possible-worlds combination of 1 and 2, where the PO dutifully prepares the backlog, only for a key stakeholder to come into planning with a completely different set of high priority requirements.

I've experienced all of the above and I'm afraid to say that none have a positive outcome.

As I mentioned in my post on in Just-in-Time Analysis, refining stories 3-4 months ahead of time risks wasting time on stories that may not be needed, or may have changed significantly, by the time work needs to start. It loses the very element of agility that Agile is supposed to bring to product, project, or programme delivery.

At the other extreme, PI Planning comes with the inherent risk that stakeholders will wait for the session to bring their requirements to the table. That's fine if they can be planned for a later iteration, but it often happens that the least well-defined requirements have the highest priority.

Since nobody wants to "waste" time on detailed refinement during what should be a high-level planning exercise, this typically results in "placeholder" stories, containing just a title (no description, value statement or acceptance criteria), being added to the backlog.

Now, you may have heard the Agile maxim that "a user story is just a placeholder for a conversation", which is true to a certain extent, but it can only work if that conversation actually takes place.

One of the biggest flaws I've seen with Big Room PI Planning is that teams are asked to commit to up to 12 weeks' work, without the conversation. If PI Planning is held immediately before the start of the first iteration, there is no time for refinement and teams are expected to just "crack on".

As a result, several stories are carried over from the first iteration for one of 3 main reasons:

1. The team had to spend the first week understanding the ask,

2. They didn't seek clarification and produced something that didn't meet the objectives,

3. They wasted 2-4 weeks just finding the right person to ask and getting their availability (more common in larger organisations than you would think).

In my opinion, the definition of "just-in-time" isn't 12 weeks or 24 hours before work starts. Even on larger programmes of work with multiple dependencies, it is still 2-6 weeks.

Productivity over Process

The other problem I have with PI Planning is the impact it has on productivity.

The fact that SAFe includes an "Innovation and Planning Iteration" smacks of mini-waterfall. It may get around the problem of refining last minute requirements but what if that "Innovation and Planning" doesn't yield the desired results? What if something changes after the planning iteration takes place and the teams suddenly have to pivot to something for which they haven't planned? They've wasted a whole iteration planning for something that may never happen and now have to spend another one planning for the new status quo. Meanwhile, actual development of "working software" has to completely stop to allow for the planning activity.

Even if an organisation doesn't have a formal "planning iteration", the planning session itself has a significant productivity cost. If the session is held in-person and teams are geographically distributed (as many are these days), the 1-2-day session becomes 2-4, as team members spend a day either side travelling to and from the venue. Add to that the tiredness that results from travel and 1-2 days of intense meetings and before you know it, a week of development has been lost, all in the name of planning.

Another thing I've observed is that it is nigh-on impossible to get a meeting with a Product Owner or Scrum Master in the 1-2 weeks leading up to a PI Planning session because they're planning for the planning session! Even typing it feels ridiculous, but it's true.

If they try and speak to stakeholders too early, they "won't know" what their priorities will be in 8-12 weeks' time (or they might but they won't consider them sufficiently urgent until nearer the time!). Try and speak to them too late and they won't be available because every other PO in the business is also trying to get a slot in their diary before PI Planning. This typically results in 1-2 weeks of back-to-back meetings that must have a significant impact on their capacity to do the "day job" of working with their development team or squad.

Continuous Refinement

Following my first, very intense, PI Planning session, where I felt more like a Court Stenographer than an Agile Consultant, I decided that, for the next iteration, we would have weekly refinement sessions with each of the Product Owners and key representatives from each dependent squad and business unit, to "tease out" their requirements on a continuous basis, rather than in the last 1-2 weeks of the Programme / Planning Increment.

During those sessions, we looked ahead to the next PI and created and refined user stories together, including the value statement and crucially the Acceptance Criteria (ACs) that would mean the story had been done.

Instead of being hurriedly dictated during PI Planning, the stories were a collaboration between the beneficiary and those that would be doing the work. Wasn't this, after all, supposed to be another one of the key benefits of Agile?

Furthermore, because the refinement sessions were scheduled as recurrent meetings, there was no last-minute scrabble for attendees' availability and stakeholders got used to thinking about their upcoming requirements. The stories to be discussed could be circulated ahead of time and any subject matter experts that weren't needed for a particular session could be excused. I found that uninviting people to meetings was a lot easier than inviting them at short notice!

In addition to the refinement sessions, I also instituted a Definition of Ready (DOR), stating the criteria that had to be met for a story to be taken into Release (Iteration) Planning (for the next 4 weeks). It had to have a meaningful title, description and ACs and the relevant assignee or team had to fully understand the story and what was being asked of them.

At the beginning of the next PI, 90% - 95% stories met the DOR for the 1st iteration but this quickly rose to 100% in subsequent releases / iterations. More importantly, "Clarification of Ask" was completely eliminated as a reason for stories being carried over.

By focusing the weekly refinement sessions on stories coming up in the next 4-week iteration and then working towards those for the other 2 releases in the PI, it enabled a telescopic view, where the stories for the next 4 weeks were very clear and those slightly further away were still known about and generally understood, but not necessarily planned out in detail.

Example of Continuous Refinement in Preparation for "Big Room" PI Planning ^© Danielle Downs 2025

This seemed to strike the right balance between giving the teams enough knowledge for longer term resource planning and estimation, without getting into too much detail, which might be wasted if there was a change in course or priorities before the start of the iteration.

As a result, the Q3 and Q4 PI Planning sessions were more of a formality, confirming what we already knew to be true and identifying a small minority of "straggler" stories that had only just been discovered, or tweaking ACs that had perhaps changed since the original refinement session because more was now known.

The sessions still took a full day, but they were held remotely (not least because the programme had scaled significantly and now involved too many people to fit in a single room) and most of the attendees just "listened in" for when their stories were being discussed, but were on hand if their input was required for other workstreams; otherwise they were able to get on with other work in the background and didn't lose an entire day, let alone a week, of productivity.

Conclusion

Operating in an Agile framework at scale and having quarterly planning sessions doesn't necessarily mean you have to have to do Big Requirements Up Front, or at least not all in one go.

Requirements definition and story refinement should be a continuous and collaborative process, involving just the necessary parties, not a reason to take an entire department and its stakeholders completely out of action for a full week.

Once regular refinement sessions are in place, teams should quickly get into a cadence of refining stories for the next iteration in the first 1-2 weeks and those for upcoming iterations in the final 1-2 weeks, depending on the iteration length.

If stories for the first iteration meet the DOR and stories for subsequent iterations are broadly defined, the PI Planning session should just become a formality of agreeing what can realistically be achieved within each increment of work and allocating it to the corresponding iteration on the backlog. Simples.