Issue Review: cloudflare/workers-sdk#9739

Summary

Header rules with multiple wildcards (*) or conflicting :splat placeholders are accepted as "valid" during build but silently fail at runtime due to JavaScript regex duplicate named capture group errors.

Findings

• Created: 2025-06-25
• Updated: 2025-10-21
• Version: Cloudflare Pages (current wrangler: 4.60.0)
• Component: Pages header rules parsing (@cloudflare/workers-shared)
• Labels: bug, pages
• Comments: 0

Key Evidence

1. No merged PRs reference this issue - Searched PRs mentioning #9739, "header rules wildcard splat", and "header validation regex" - none found.

2. No changelog entries - Issue #9739 not mentioned in wrangler changelog. No recent changes related to multiple wildcards or duplicate splat validation.

3. Source code confirms the bug still exists:

   • packages/workers-shared/asset-worker/src/utils/rules-engine.ts:43:

     rule = rule.split("*").map(escapeRegex).join("(?<splat>.*)");
     

     Every * becomes (?<splat>.*). With two * chars (e.g., https://*.pages.dev/*), this creates duplicate named capture groups which throws a RegExp error.

   • packages/workers-shared/asset-worker/src/utils/rules-engine.ts:80-83:

     try {
         const regExp = generateRuleRegExp(rule);
         return [{ crossHost, regExp }, match];
     } catch {}
     

     The error is silently caught and the rule is filtered out.

   • packages/workers-shared/utils/configuration/parseHeaders.ts has no validation for multiple wildcards in paths.

4. No test coverage - Reviewed rules-engine.test.ts and parseHeaders.invalid.test.ts. No test cases for multiple wildcards or duplicate placeholder names.

Recommendation

Status: KEEP OPEN

Reasoning: The bug is confirmed in the current codebase. The root cause is clear: generateRuleRegExp() creates invalid regexes with duplicate named capture groups when paths contain multiple * characters, and the error is silently swallowed. This matches exactly what the issue reporter described.

Action: Implement validation in parseHeaders() to reject paths with multiple wildcards, OR modify generateRuleRegExp() to use unique capture group names (e.g., (?<splat1>.*), (?<splat2>.*)).

---

Root Cause Analysis

Problem 1: Multiple Wildcards Create Invalid Regex

Location: packages/workers-shared/asset-worker/src/utils/rules-engine.ts:41-66

export const generateRuleRegExp = (rule: string) => {
  // Create :splat capturer then escape.
  rule = rule.split("*").map(escapeRegex).join("(?<splat>.*)");  // BUG: line 43
  // ...
  return RegExp(rule);  // THROWS when multiple splats
};

When a rule like https://*.pages.dev/* is processed:

1. Split by *: ["https://", ".pages.dev/", ""]
2. Escape: ["https:\\/\\/", "\\.pages\\.dev\\/", ""]
3. Join with (?<splat>.*): "https:\\/\\/(?<splat>.*)\\.pages\\.dev\\/(?<splat>.*)"
4. RegExp() throws: "Duplicate capture group name"

Problem 2: Silent Error Suppression

Location: packages/workers-shared/asset-worker/src/utils/rules-engine.ts:76-88

const compiledRules = Object.entries(rules)
  .map(([rule, match]) => {
    const crossHost = rule.startsWith("https://");
    try {
      const regExp = generateRuleRegExp(rule);
      return [{ crossHost, regExp }, match];
    } catch {}  // BUG: Silently swallows the error
  })
  .filter((value) => value !== undefined) as [...];  // Invalid rules disappear

Problem 3: No Validation in Parser

Location: packages/workers-shared/utils/configuration/parseHeaders.ts:69

The validateUrl() function called at line 69 only validates URL format, not wildcard count:

• Checks for https:// protocol
• Checks for port numbers
• Does NOT check for multiple * characters

Problem 4: :splat Placeholder Conflicts

When a user defines a rule like https://*.pages.dev/:splat, the generated regex has:

• (?<splat>.*) from the * wildcard
• An attempt to create (?<splat>[^/]+) from the :splat placeholder

This also fails due to duplicate capture group names.

---

Proposed Solution

Option A: Validate During Parsing (Recommended)

Add validation in parseHeaders.ts to reject paths with multiple wildcards:

File: packages/workers-shared/utils/configuration/parseHeaders.ts

// Add after line 8
const MULTIPLE_WILDCARDS_REGEX = /\*.*\*/;
const SPLAT_PLACEHOLDER_REGEX = /:splat\b/i;

// Add validation function
function validateWildcards(path: string): string | undefined {
  const wildcardCount = (path.match(/\*/g) || []).length;
  const hasSplatPlaceholder = SPLAT_PLACEHOLDER_REGEX.test(path);
  
  if (wildcardCount > 1) {
    return "Multiple wildcards (*) are not supported in a single rule. Use a single wildcard or create separate rules.";
  }
  
  if (wildcardCount > 0 && hasSplatPlaceholder) {
    return "Cannot combine wildcard (*) with :splat placeholder. The wildcard already captures as :splat.";
  }
  
  return undefined;
}

// In the main parsing loop, after validateUrl() call (around line 69):
const [path, pathError] = validateUrl(line, false, true);
if (pathError) {
  invalid.push({ line, lineNumber: i + 1, message: pathError });
  rule = undefined;
  continue;
}

// Add wildcard validation
const wildcardError = validateWildcards(path as string);
if (wildcardError) {
  invalid.push({ line, lineNumber: i + 1, message: wildcardError });
  rule = undefined;
  continue;
}

Option B: Use Unique Capture Group Names (Alternative)

Modify generateRuleRegExp() to use numbered capture groups:

File: packages/workers-shared/asset-worker/src/utils/rules-engine.ts

export const generateRuleRegExp = (rule: string) => {
  // Create numbered :splat capturers
  let splatIndex = 0;
  rule = rule.split("*").map(escapeRegex).join(() => `(?<splat${splatIndex++}>.*)`);
  
  // ... rest of function
};

// Update replacer function to handle multiple splats
export const replacer = (str: string, replacements: Replacements) => {
  // Combine all splat values for backward compatibility
  const splatValues: string[] = [];
  for (const [key, value] of Object.entries(replacements)) {
    if (key.startsWith('splat')) {
      splatValues.push(value);
    }
  }
  if (splatValues.length > 0) {
    replacements.splat = splatValues.join('/');
  }
  
  for (const [replacement, value] of Object.entries(replacements)) {
    str = str.replaceAll(`:${replacement}`, value);
  }
  return str;
};

Recommendation: Option A is preferred because:

1. It catches errors early at parse time with clear error messages
2. It follows the existing pattern of validation in parseHeaders
3. The documented Cloudflare behavior likely expects a single splat capture

---

Implementation Difficulty: Easy

Justification:

1. The fix is localized to 1-2 files
2. Clear validation logic with no complex edge cases
3. Existing test infrastructure can be extended
4. No breaking changes to the public API (invalid rules were silently ignored anyway)

---

Files to Modify

File	Changes
packages/workers-shared/utils/configuration/parseHeaders.ts	Add wildcard validation logic
packages/workers-shared/utils/tests/parseHeaders.invalid.test.ts	Add test cases for multiple wildcards
packages/workers-shared/asset-worker/src/utils/rules-engine.ts	(Optional) Add warning log when regex creation fails
packages/workers-shared/asset-worker/tests/rules-engine.test.ts	Add test cases for edge cases

---

Testing Recommendations

Unit Tests to Add

File: packages/workers-shared/utils/tests/parseHeaders.invalid.test.ts

test("parseHeaders should reject multiple wildcards", () => {
  const input = `
    https://*.pages.dev/*
      X-Robots-Tag: noindex
  `;
  const result = parseHeaders(input);
  expect(result).toEqual({
    rules: [],
    invalid: [
      {
        line: "https://*.pages.dev/*",
        lineNumber: 2,
        message: "Multiple wildcards (*) are not supported in a single rule. Use a single wildcard or create separate rules.",
      },
    ],
  });
});

test("parseHeaders should reject wildcard combined with :splat placeholder", () => {
  const input = `
    https://*.pages.dev/:splat
      Test: value
  `;
  const result = parseHeaders(input);
  expect(result).toEqual({
    rules: [],
    invalid: [
      {
        line: "https://*.pages.dev/:splat",
        lineNumber: 2,
        message: "Cannot combine wildcard (*) with :splat placeholder. The wildcard already captures as :splat.",
      },
    ],
  });
});

E2E Test Suggestions

1. Deploy a Pages project with a _headers file containing multiple wildcards
2. Verify the build output shows an error (after fix)
3. Verify valid single-wildcard rules still work correctly

Manual Testing

1. Create a _headers file with:

   https://*.pages.dev/*
     X-Robots-Tag: noindex
   

2. Run wrangler pages dev or deploy
3. Verify error message appears (after fix)
4. Confirm rule is NOT silently accepted

---

Suggested Comment

Thank you for the detailed bug report! This issue has been confirmed and is still present in the current codebase.

Root Cause: The generateRuleRegExp() function in rules-engine.ts converts each * to (?<splat>.*), creating duplicate named capture groups when multiple wildcards are present. JavaScript's RegExp constructor throws on duplicate names, and this error is silently caught, causing the rule to be dropped.

Proposed Fix: Add validation in parseHeaders() to reject paths with multiple wildcards or conflicting :splat placeholders, with clear error messages.

The fix is straightforward - primarily changes to parseHeaders.ts with corresponding test updates.