Redaction¶

Redaction is httptape's most distinctive feature. Sensitive data (secrets, PII, credentials) is redacted or replaced with deterministic fakes before it touches disk. This is the second R in httptape's Record, Redact, Replay pipeline.

The redaction pipeline is implemented in Go as a Pipeline of SanitizeFunc transformations (the Go types use "sanitize" terminology). Each function receives a Tape and returns a (possibly modified) copy. Functions are applied in order.

type SanitizeFunc func(Tape) Tape

type Pipeline struct { /* ... */ }
func NewPipeline(funcs ...SanitizeFunc) *Pipeline
func (p *Pipeline) Sanitize(t Tape) Tape

The Pipeline implements the Sanitizer interface, which the Recorder accepts:

type Sanitizer interface {
    Sanitize(Tape) Tape
}

Building a redaction pipeline¶

sanitizer := httptape.NewPipeline(
    httptape.RedactHeaders(),
    httptape.RedactQueryParams(),
    httptape.RedactBodyPaths("$.password", "$.ssn"),
    httptape.FakeFields("my-seed", "$.email", "$.user_id"),
)

rec := httptape.NewRecorder(store,
    httptape.WithSanitizer(sanitizer),
)

Functions are applied in order. In this example: headers are redacted first, then URL query parameters are redacted, then body fields are redacted, then remaining fields get deterministic fakes.

RedactHeaders¶

Replaces header values with "[REDACTED]" in both request and response headers.

// Redact the default sensitive headers:
httptape.RedactHeaders()

Default sensitive headers: - Authorization -- bearer tokens, basic auth - Cookie -- session tokens - Set-Cookie -- server-set sessions - X-Api-Key -- API key auth - Proxy-Authorization -- proxy auth - X-Forwarded-For -- client IPs (PII)

To redact specific headers:

httptape.RedactHeaders("Authorization", "X-Custom-Secret", "X-Internal-Token")

Header matching is case-insensitive per the HTTP spec.

You can retrieve the default list programmatically:

defaults := httptape.DefaultSensitiveHeaders()
// ["Authorization", "Cookie", "Set-Cookie", "X-Api-Key", "Proxy-Authorization", "X-Forwarded-For"]

RedactBodyPaths¶

Redacts fields within JSON request and response bodies at specified paths.

httptape.RedactBodyPaths("$.password", "$.user.ssn", "$.tokens[*].value")

Path syntax¶

Paths use a JSONPath-like syntax:

Redaction behavior¶

Redacted values are type-aware:

If the body is not valid JSON, it is left unchanged (no error). Missing paths are silently skipped.

Example¶

Input body:

{
  "username": "alice",
  "password": "s3cret",
  "profile": {
    "ssn": "123-45-6789"
  }
}

With RedactBodyPaths("$.password", "$.profile.ssn"):

{
  "username": "alice",
  "password": "[REDACTED]",
  "profile": {
    "ssn": "[REDACTED]"
  }
}

FakeFields¶

Replaces field values with deterministic fakes derived from HMAC-SHA256. The same seed and input value always produce the same fake output, preserving cross-fixture consistency.

httptape.FakeFields("my-project-seed",
    "$.user.email",
    "$.user.id",
    "$.tokens[*].value",
)

The seed parameter¶

The first argument is a project-level seed used as the HMAC key. Different seeds produce different fakes. The same seed and input always produce the same output.

The seed must be non-empty and unique to your project. Treat it as a moderately sensitive value: anyone who knows the seed can predict the fake output for any input. Do not use an empty string, a default placeholder, or a seed shared across unrelated projects. Store it alongside other project configuration (e.g. environment variables), not in public source code.

Faking strategies¶

The fake value depends on the detected type of the original value:

Example¶

Input body:

{
  "user": {
    "email": "alice@company.com",
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "name": "Alice Smith"
  }
}

With FakeFields("my-seed", "$.user.email", "$.user.id", "$.user.name"):

{
  "user": {
    "email": "user_7f3a2b1c@example.com",
    "id": "7f3a2b1c-4d5e-5f60-8a9b-c0d1e2f3a4b5",
    "name": "fake_7f3a2b1c"
  }
}

The key property: if alice@company.com appears in another fixture, it will be faked to the same value. This preserves relational consistency across your fixture set.

Typed fakers¶

FakeFields auto-detects the faking strategy from each value's runtime type. When you need explicit control -- for example, to force a generic-looking string into the credit-card format, or to fake a numeric ID as a fixed-length digit string -- use the typed-faker API.

The contract is the Faker interface:

type Faker interface {
    Fake(seed string, original any) any
}

Implementations take an HMAC seed and the original JSON value (already decoded by encoding/json, so strings are string, numbers are float64, booleans are bool, nulls are nil, objects are map[string]any, arrays are []any). They must be deterministic: the same seed and original must always produce the same fake.

Typed fakers are wired into a pipeline with FakeFieldsWith, which takes a seed and a path-to-Faker map:

sanitizer := httptape.NewPipeline(
    httptape.FakeFieldsWith("my-seed", map[string]httptape.Faker{
        "$.user.email":  httptape.EmailFaker{},
        "$.user.phone":  httptape.PhoneFaker{},
        "$.card.number": httptape.CreditCardFaker{},
        "$.card.cvv":    httptape.NumericFaker{Length: 3},
    }),
)

Path syntax is identical to RedactBodyPaths and FakeFields. Invalid JSON bodies, missing paths, and invalid path strings are silently skipped (no error). All twelve built-in fakers are constructed as struct literals; none has a NewXFaker constructor.

Auto-detect vs. typed -- when to use each¶

Built-in fakers -- redaction-style¶

These fakers replace values without preserving any information from the original. Use them when the original content is sensitive enough that even a deterministic transform of it should not appear in fixtures.

RedactedFaker¶

Replaces strings with "[REDACTED]", numbers with 0, and booleans with false. Other types (nil, objects, arrays) pass through unchanged. Equivalent to the leaf behavior of RedactBodyPaths, but addressable through the typed-faker map so you can mix it with other fakers in a single FakeFieldsWith call.

httptape.FakeFieldsWith("seed", map[string]httptape.Faker{
    "$.password": httptape.RedactedFaker{},
    "$.email":    httptape.EmailFaker{},
})

FixedFaker¶

Always returns its Value field, ignoring both seed and original. Useful for stamping a known sentinel into a field (e.g., "status": "active") so downstream tests can assert against it.

httptape.FixedFaker{Value: "active"}
httptape.FixedFaker{Value: float64(1)}
httptape.FixedFaker{Value: true}

Value is any, so the encoded JSON type follows Go's encoding/json rules.

Built-in fakers -- generic deterministic¶

These fakers produce a deterministic value derived from HMAC-SHA256(seed, original), with no PII shape. Use them when you need consistency across fixtures but do not need the output to look like any particular format.

HMACFaker¶

Mirrors the auto-detect default for generic strings and numbers. Strings become "fake_<8-hex>"; numbers become a positive integer in [1, 2^31-1]; booleans, nulls, objects, and arrays pass through unchanged.

httptape.HMACFaker{}
// "abc"        -> "fake_a1b2c3d4"
// float64(42)  -> 1734567890

NumericFaker¶

Generates a string of Length HMAC-derived digits. Useful for fixed-width numeric IDs (CVVs, OTPs, account numbers) where digit-count matters.

httptape.NumericFaker{Length: 3}   // CVV
httptape.NumericFaker{Length: 16}  // account number

If the input is not a string, it is returned unchanged. Output length always equals Length, even if Length exceeds 32 (the HMAC is re-chained).

PatternFaker¶

Fills a template where # becomes a digit, ? becomes a lowercase letter, and any other character is copied literally.

httptape.PatternFaker{Pattern: "###-##-####"}     // SSN-shaped
httptape.PatternFaker{Pattern: "??-#####"}         // 2 letters, dash, 5 digits
httptape.PatternFaker{Pattern: "ORDER-####-????"} // mixed literal + variable

If the input is not a string, it is returned unchanged.

PrefixFaker¶

Generates "<Prefix><16-hex>". Useful when an upstream issues namespaced identifiers (e.g., cust_*, order_*) and downstream tests look at the prefix.

httptape.PrefixFaker{Prefix: "cust_"}   // "cust_a1b2c3d4e5f60718"
httptape.PrefixFaker{Prefix: "order_"}  // "order_a1b2c3d4e5f60718"

If the input is not a string, it is returned unchanged.

DateFaker¶

Generates a date string formatted with Format (Go reference layout; defaults to "2006-01-02" when empty). The date is drawn deterministically from a ~100-year window starting at 2000-01-01.

httptape.DateFaker{}                       // "2042-09-13"
httptape.DateFaker{Format: "2006-01-02"}   // "2042-09-13"
httptape.DateFaker{Format: time.RFC3339}   // "2042-09-13T00:00:00Z"

If the input is not a string, it is returned unchanged.

Built-in fakers -- PII-shaped¶

These fakers preserve a recognizable shape (so downstream parsers do not break) while replacing the underlying content. They are the right choice for fields whose format matters to consumers (clients that validate emails, payment processors that check Luhn, address forms that expect a US zip).

EmailFaker¶

Replaces strings with "user_<8-hex>@example.com". Non-string inputs pass through unchanged.

httptape.EmailFaker{}
// "alice@corp.com" -> "user_a1b2c3d4@example.com"

PhoneFaker¶

Replaces digits in the input with HMAC-derived digits while preserving every non-digit character (spaces, dashes, parentheses, plus signs). Output length always equals input length.

httptape.PhoneFaker{}
// "+1 (555) 123-4567" -> "+1 (937) 481-2056"
// "555-1234"          -> "938-1742"

If the input is not a string, it is returned unchanged.

CreditCardFaker¶

Generates a 16-digit number formatted as XXXX-XXXX-XXXX-XXXX. The first 6 digits (issuer prefix) are taken from the original; if the original has fewer than 6 digits, the prefix 400000 is used. The middle 9 digits are HMAC-derived; the last digit is a valid Luhn check digit.

httptape.CreditCardFaker{}
// "4532-1234-5678-9012" -> "4532-12<derived>-<luhn>"

If the input is not a string, it is returned unchanged.

NameFaker¶

Picks a first name and a last name from internal fixed lists using two HMAC bytes. Output is "<First> <Last>".

httptape.NameFaker{}
// "Alice Johnson" -> "Olivia Martinez" (deterministic for that seed+input)

If the input is not a string, it is returned unchanged.

AddressFaker¶

Generates a US-style address: "<number> <street> <suffix>, <city>, <ST> <zip>". House number is in [1, 9999]; zip is 5 digits; city, state, and street components are picked from internal fixed lists.

httptape.AddressFaker{}
// "123 Main St, Anytown, CA 90210" -> "8421 Cedar Drive, Salem, NV 30418"

If the input is not a string, it is returned unchanged.

Custom fakers¶

Anything that satisfies the Faker interface works. A typical use case is wrapping an existing data source (a list of canonical fake company names, a generator of valid IBANs, etc.) so the recorded fixtures are coherent with the rest of your test data.

import (
    "crypto/hmac"
    "crypto/sha256"
)

type CompanyFaker struct {
    Names []string
}

func (f CompanyFaker) Fake(seed string, original any) any {
    s, ok := original.(string)
    if !ok || len(f.Names) == 0 {
        return original
    }
    // Deterministic pick using the HMAC of seed||s.
    mac := hmac.New(sha256.New, []byte(seed))
    mac.Write([]byte(s))
    h := mac.Sum(nil)
    idx := int(h[0]) % len(f.Names)
    return f.Names[idx]
}

sanitizer := httptape.NewPipeline(
    httptape.FakeFieldsWith("my-seed", map[string]httptape.Faker{
        "$.employer": CompanyFaker{Names: []string{"Acme", "Globex", "Initech"}},
    }),
)

Make sure your implementation is deterministic (same seed + original always produces the same output) and does not mutate original -- httptape passes the value pulled out of json.Unmarshal directly.

RedactQueryParams¶

Replaces URL query-parameter values with "[REDACTED]" in t.Request.URL. Userinfo credentials (user:pass@host) are unconditionally stripped as well, regardless of which parameter names are configured.

// Redact the default sensitive query parameters:
httptape.RedactQueryParams()

// Redact specific named parameters:
httptape.RedactQueryParams("api_key", "access_token", "sig")

Default sensitive parameters (used when called with no arguments): - api_key -- generic API key authentication - access_token -- OAuth access tokens - token -- generic tokens - secret -- generic secrets - password -- plaintext passwords in URLs - sig -- presigned URL signatures (short form) - signature -- presigned URL signatures (long form) - X-Amz-Signature -- AWS presigned URL signature - X-Goog-Signature -- Google Cloud presigned URL signature

You can retrieve the default list programmatically:

defaults := httptape.DefaultSensitiveQueryParams()

Parameter name matching is case-sensitive per RFC 3986. api_key does not match API_KEY. This is intentional: query keys are case-sensitive by specification, and silent case-folding would risk redacting the wrong parameter.

Fail-closed behavior: if the URL's query string contains malformed percent-encoding (e.g., ?api_key=ab%ZZcd), the entire query string is replaced with [REDACTED] rather than passed through. This prevents cleartext secrets from reaching disk when a value cannot be safely decoded.

Fragment handling: OAuth implicit-flow tokens and some presigned-URL schemes embed credentials in the URL fragment (#access_token=...). RedactQueryParams applies the same name-based redaction to fragment key=value pairs. On a fragment parse error the entire fragment is replaced with [REDACTED].

Byte-stable on no-match: when no configured parameter name matches and no userinfo is present, the URL is returned byte-identical (no re-encoding, no key reordering). When a match occurs or userinfo is stripped, the query string is re-encoded and keys may be re-sorted alphabetically -- this is safe for all built-in matchers, which compare by key/value map, not by string order.

Headers, body, and response are left byte-identical after URL sanitization.

If the URL fails to parse structurally (invalid scheme, etc.), it is returned unchanged (silent skip, consistent with body sanitization behavior).

FakeQueryParams¶

Replaces URL query-parameter values with deterministic HMAC-SHA256 fakes. Userinfo credentials (user:pass@host) are unconditionally stripped as well.

httptape.FakeQueryParams("my-project-seed", "api_key", "access_token")

The first argument is the project-level seed (same as FakeFields). The same seed and original value always produce the same fake output. Different seeds produce different outputs. Use the same seed as the rest of your sanitization pipeline to maintain cross-fixture consistency.

The fake value is computed as fakeString(HMAC-SHA256(seed, original_value)), producing a "fake_<hex>" string. This is the same algorithm used by HMACFaker for generic string fields.

All the same behavioral guarantees as RedactQueryParams apply: case-sensitive name matching, fail-closed on malformed query/fragment, unconditional userinfo stripping, byte-stable on no-match.

The declarative config equivalent is the fake_query action. Unlike redact_query, the params field is required for fake_query -- an empty list is rejected as a likely misconfiguration.

Combining redaction and faking¶

Order matters. Typically, redact first (remove things that should be gone entirely), then fake (replace things that need consistent stand-in values):

sanitizer := httptape.NewPipeline(
    // Step 1: Remove sensitive headers entirely
    httptape.RedactHeaders(),

    // Step 2: Redact body fields that should be blank
    httptape.RedactBodyPaths("$.password", "$.credit_card.number"),

    // Step 3: Replace PII with deterministic fakes
    httptape.FakeFields("my-seed",
        "$.user.email",
        "$.user.phone",
        "$.user.id",
    ),
)

CLI and Docker¶

The record and proxy commands fail closed: without --config, a built-in safe sanitization pipeline (default sensitive headers, query params, and URL userinfo) is applied automatically and a warning naming what is redacted is printed to stderr. The safe default does not redact request or response bodies — use --config with redact_body or fake rules to cover secrets in bodies. Supply --config to replace the safe default with a custom pipeline; use --unsafe-raw to disable all sanitization (not recommended, prints a loud warning).

# Safe-by-default (no config needed — headers + query params + userinfo are redacted)
httptape record --upstream https://api.example.com --fixtures ./mocks

# Custom rules (config replaces the safe default)
httptape record --upstream https://api.example.com --fixtures ./mocks --config redact.json

# Proxy with custom rules (applied to L2/disk cache only)
httptape proxy --upstream https://api.example.com --fixtures ./cache --config redact.json

See CLI and Docker for full usage.

Declarative configuration¶

Instead of building pipelines in code, you can define redaction rules in a JSON config file. See Config for details.

{
  "version": "1",
  "rules": [
    { "action": "redact_headers" },
    { "action": "redact_query" },
    { "action": "redact_body", "paths": ["$.password"] },
    { "action": "fake", "seed": "my-seed", "paths": ["$.user.email"] }
  ]
}

The fake action also accepts a fields map that selects a typed faker per path -- the JSON-config equivalent of FakeFieldsWith. The fake_query action is the config equivalent of FakeQueryParams. See Config for the full config syntax and all supported actions.

SSE event redaction¶

For SSE (Server-Sent Events) responses, httptape provides two SanitizeFunc constructors that operate on individual event payloads. Each event's Data field is treated as an independent JSON body, so the same path syntax applies as RedactBodyPaths and FakeFields.

These functions are no-ops for non-SSE tapes, so they compose safely in a pipeline that handles both regular and SSE responses.

RedactSSEEventData¶

Redacts fields within each SSE event's JSON data:

httptape.RedactSSEEventData("$.choices[*].delta.content", "$.usage.prompt_tokens")

Example: an LLM streaming response where each event looks like:

{"id":"chatcmpl-1","choices":[{"delta":{"content":"The user's SSN is 123-45-6789"}}]}

After RedactSSEEventData("$.choices[*].delta.content"):

{"id":"chatcmpl-1","choices":[{"delta":{"content":"[REDACTED]"}}]}

Each event is redacted independently. Non-JSON event data (e.g., [DONE]) is left unchanged.

FakeSSEEventData¶

Replaces fields within each SSE event's JSON data with deterministic fakes:

httptape.FakeSSEEventData("my-seed", "$.user.email", "$.user.name")

This uses the same HMAC-SHA256 faking strategy as FakeFields. The same seed and input always produce the same fake, so cross-event consistency is preserved.

Complete pipeline for LLM streaming¶

A typical pipeline for recording LLM API traffic with streaming redaction:

sanitizer := httptape.NewPipeline(
    // Step 1: Redact auth headers.
    httptape.RedactHeaders("Authorization", "X-Api-Key"),

    // Step 2: Redact sensitive fields in regular (non-SSE) response bodies.
    httptape.RedactBodyPaths("$.api_key"),

    // Step 3: Redact PII from SSE event payloads.
    httptape.RedactSSEEventData("$.choices[*].delta.content"),

    // Step 4: Fake user identifiers in SSE events with deterministic values.
    httptape.FakeSSEEventData("my-seed", "$.user.email", "$.user.id"),
)

rec := httptape.NewRecorder(store, httptape.WithSanitizer(sanitizer))

The order is: headers first, regular body paths, SSE event redaction, SSE event faking. SSE-specific functions are no-ops for non-SSE tapes, and RedactBodyPaths/FakeFields are no-ops for SSE tapes (since SSE tapes have nil Body), so all functions coexist safely in one pipeline.

Custom sanitize functions¶

You can write your own SanitizeFunc and add it to the pipeline:

func maskIPAddresses() httptape.SanitizeFunc {
    return func(t httptape.Tape) httptape.Tape {
        // Your custom transformation logic
        // Remember: do not mutate the input tape -- copy fields you modify
        return t
    }
}

sanitizer := httptape.NewPipeline(
    httptape.RedactHeaders(),
    maskIPAddresses(),
)

See also¶

• Config -- declarative JSON configuration
• Recording -- attaching the redaction pipeline to recorders
• Proxy Mode -- redaction in proxy mode (L2 writes only)
• API Reference -- full type signatures