[Regression] Min duration under steady state

@nv-alicheng

The detector computes a minimum window wall-time per candidate plateau, from that plateau's own data, then uses it to pick which plateau becomes the reported steady state. So the certified metric value depends on a (self-referenced) data-derived threshold, not a published constant. It also diverges from the ratified fixed minimum in the policy 6.2
I think the statistical intent is sound but the mechanism may not be appropriate for the benchmarks. While it's understandable to extend/apply min duration to steady-state plateaus, if done, it'd make sense to use the fixed duration requirement, i.e 600s minimum per plateau.

The specific section of reference is in the min_steady_duration routine:

endpoints/src/inference_endpoint/metrics/steady_state_diagnostics.py

Lines 1633 to 1650 in f1100cf

    
           cov_b = cov(tpot_p50) if len(tpot_p50) >= 2 else 0.0 
        
           spans = [ 
        
               (sp.last_issue_ns - sp.first_issue_ns) / 1e9 
        
               for sp in window 
        
               if sp.last_issue_ns > sp.first_issue_ns >= 0 
        
           ] 
        
           tau_sp = median(spans) if spans else 0.0 
        
           lat = pooled(series, lo, hi, "latency_ns") 
        
           l_p90 = percentile_lower(sorted(lat), 0.90) / 1e9 if lat else 0.0 
        
           kstar = max(math.ceil((CI_Z_95 * cov_b / MIN_DUR_EPS) ** 2), MIN_DUR_KSTAR_FLOOR) 
        
           # The precision term binds only when noise demands more batches than the 
        
           # trend floor. At k* == MIN_TREND_N, the trend requirement is already met. 
        
           # Then relaxation and floor control duration. This keeps a clean minimal 
        
           # plateau valid when it meets the wall-time floor. 
        
           t_prec = kstar * tau_sp if kstar > MIN_DUR_KSTAR_FLOOR else 0.0 
        
           t_relax = MIN_DUR_RELAX_MULT * l_p90 
        
           min_s = max(t_prec, t_relax, MIN_DUR_FLOOR_S)

Given the parameters up to the min time calculation are self-referenced (w.r.t the plateau itself), each plateau is judged against a bar derived from its own noise/latency; plateaus aren't compared on a common yardstick.

In the meantime, we can default enforce_min_duration=False

endpoints/src/inference_endpoint/metrics/steady_state_diagnostics.py

Line 2147 in f1100cf

dest="enforce_min_duration",

until any dynamic threshold is ratified in the policies.

@nv-alicheng @arekay-nv @mrmhodak @nvashutoshd @hans-intel

@attafosu

@attafosu - enabling enforce_min_duration to false would admit steady state windows shorter than 600s - is that the intent? Currently, the max would ensure that each candidate window is at least 600s.

@arekay-nv

@arekay-nv I'm thinking more of using the fixed 600s min duration requirement instead of the max over the 3 parameters (t_prec, t_relax, 600s). The other two parameters are calculated from the plateau itself, and I imagine those would be quite noisy, and when either is higher than 600s, it may lead to a rejection of an otherwise 600s min-duration compliant plateau.

On the enforce_min_duration=False, you're right... I just noticed that'd relax the entire min duration constraint. Perhaps we can make the 600s the only gate, while reporting the other two parameters for informational purposes only...

Leaving this open for any followup. If no update in the next two cycles, we can close it.

Navigation Menu

Repository navigation

[Regression] Min duration under steady state #525

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	cov_b = cov(tpot_p50) if len(tpot_p50) >= 2 else 0.0
	spans = [
	(sp.last_issue_ns - sp.first_issue_ns) / 1e9
	for sp in window
	if sp.last_issue_ns > sp.first_issue_ns >= 0
	]
	tau_sp = median(spans) if spans else 0.0
	lat = pooled(series, lo, hi, "latency_ns")
	l_p90 = percentile_lower(sorted(lat), 0.90) / 1e9 if lat else 0.0

	kstar = max(math.ceil((CI_Z_95 * cov_b / MIN_DUR_EPS) ** 2), MIN_DUR_KSTAR_FLOOR)
	# The precision term binds only when noise demands more batches than the
	# trend floor. At k* == MIN_TREND_N, the trend requirement is already met.
	# Then relaxation and floor control duration. This keeps a clean minimal
	# plateau valid when it meets the wall-time floor.
	t_prec = kstar * tau_sp if kstar > MIN_DUR_KSTAR_FLOOR else 0.0
	t_relax = MIN_DUR_RELAX_MULT * l_p90
	min_s = max(t_prec, t_relax, MIN_DUR_FLOOR_S)

[Regression] Min duration under steady state #525

Description

Activity

arekay-nv commented on Oct 11, 2026

attafosu commented on Oct 12, 2026

arekay-nv commented on Nov 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions