fix(#188): sanitize Discord error logs to prevent secret exposure
P1 SECURITY FIX - Prevents credential leakage through error logs Changes: 1. Created comprehensive log sanitization utility (log-sanitizer.ts) - Detects and redacts API keys, tokens, passwords, emails - Deep object traversal with circular reference detection - Preserves Error objects and non-sensitive data - Performance optimized (<100ms for 1000+ keys) 2. Integrated sanitizer into Discord service error logging - All error logs automatically sanitized before Discord broadcast - Prevents bot tokens, API keys, passwords from being exposed 3. Comprehensive test suite (32 tests, 100% passing) - Tests all sensitive pattern detection - Verifies deep object sanitization - Validates performance requirements Security Patterns Redacted: - API keys (sk_live_*, pk_test_*) - Bearer tokens and JWT tokens - Discord bot tokens - Authorization headers - Database credentials - Email addresses - Environment secrets - Generic password patterns Test Coverage: 97.43% (exceeds 85% requirement) Fixes #188 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -7,6 +7,7 @@ import type { ChatMessage, ChatCommand } from "../interfaces";
|
|||||||
|
|
||||||
// Mock discord.js Client
|
// Mock discord.js Client
|
||||||
const mockReadyCallbacks: Array<() => void> = [];
|
const mockReadyCallbacks: Array<() => void> = [];
|
||||||
|
const mockErrorCallbacks: Array<(error: Error) => void> = [];
|
||||||
const mockClient = {
|
const mockClient = {
|
||||||
login: vi.fn().mockImplementation(async () => {
|
login: vi.fn().mockImplementation(async () => {
|
||||||
// Trigger ready callback when login is called
|
// Trigger ready callback when login is called
|
||||||
@@ -14,7 +15,11 @@ const mockClient = {
|
|||||||
return Promise.resolve();
|
return Promise.resolve();
|
||||||
}),
|
}),
|
||||||
destroy: vi.fn().mockResolvedValue(undefined),
|
destroy: vi.fn().mockResolvedValue(undefined),
|
||||||
on: vi.fn(),
|
on: vi.fn().mockImplementation((event: string, callback: (error: Error) => void) => {
|
||||||
|
if (event === "error") {
|
||||||
|
mockErrorCallbacks.push(callback);
|
||||||
|
}
|
||||||
|
}),
|
||||||
once: vi.fn().mockImplementation((event: string, callback: () => void) => {
|
once: vi.fn().mockImplementation((event: string, callback: () => void) => {
|
||||||
if (event === "ready") {
|
if (event === "ready") {
|
||||||
mockReadyCallbacks.push(callback);
|
mockReadyCallbacks.push(callback);
|
||||||
@@ -73,8 +78,9 @@ describe("DiscordService", () => {
|
|||||||
process.env.DISCORD_CONTROL_CHANNEL_ID = "test-channel-id";
|
process.env.DISCORD_CONTROL_CHANNEL_ID = "test-channel-id";
|
||||||
process.env.DISCORD_WORKSPACE_ID = "test-workspace-id";
|
process.env.DISCORD_WORKSPACE_ID = "test-workspace-id";
|
||||||
|
|
||||||
// Clear ready callbacks
|
// Clear callbacks
|
||||||
mockReadyCallbacks.length = 0;
|
mockReadyCallbacks.length = 0;
|
||||||
|
mockErrorCallbacks.length = 0;
|
||||||
|
|
||||||
const module: TestingModule = await Test.createTestingModule({
|
const module: TestingModule = await Test.createTestingModule({
|
||||||
providers: [
|
providers: [
|
||||||
@@ -533,4 +539,117 @@ describe("DiscordService", () => {
|
|||||||
process.env.DISCORD_WORKSPACE_ID = "test-workspace-id";
|
process.env.DISCORD_WORKSPACE_ID = "test-workspace-id";
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("Error Logging Security", () => {
|
||||||
|
it("should sanitize sensitive data in error logs", () => {
|
||||||
|
const loggerErrorSpy = vi.spyOn((service as any).logger, "error");
|
||||||
|
|
||||||
|
// Simulate an error with sensitive data
|
||||||
|
const errorWithSecrets = new Error("Connection failed");
|
||||||
|
(errorWithSecrets as any).config = {
|
||||||
|
headers: {
|
||||||
|
Authorization: "Bearer secret_token_12345",
|
||||||
|
},
|
||||||
|
};
|
||||||
|
(errorWithSecrets as any).token = "MTk4NjIyNDgzNDcxOTI1MjQ4.Cl2FMQ.ZnCjm1XVW7vRze4b7Cq4se7kKWs";
|
||||||
|
|
||||||
|
// Trigger error event handler
|
||||||
|
expect(mockErrorCallbacks.length).toBeGreaterThan(0);
|
||||||
|
mockErrorCallbacks[0]?.(errorWithSecrets);
|
||||||
|
|
||||||
|
// Verify error was logged
|
||||||
|
expect(loggerErrorSpy).toHaveBeenCalled();
|
||||||
|
|
||||||
|
// Get the logged error
|
||||||
|
const loggedArgs = loggerErrorSpy.mock.calls[0];
|
||||||
|
const loggedError = loggedArgs[1];
|
||||||
|
|
||||||
|
// Verify sensitive data was redacted
|
||||||
|
expect(loggedError.config.headers.Authorization).toBe("[REDACTED]");
|
||||||
|
expect(loggedError.token).toBe("[REDACTED]");
|
||||||
|
expect(loggedError.message).toBe("Connection failed");
|
||||||
|
expect(loggedError.name).toBe("Error");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should not leak bot token in error logs", () => {
|
||||||
|
const loggerErrorSpy = vi.spyOn((service as any).logger, "error");
|
||||||
|
|
||||||
|
// Simulate an error with bot token in message
|
||||||
|
const errorWithToken = new Error(
|
||||||
|
"Discord authentication failed with token MTk4NjIyNDgzNDcxOTI1MjQ4.Cl2FMQ.ZnCjm1XVW7vRze4b7Cq4se7kKWs"
|
||||||
|
);
|
||||||
|
|
||||||
|
// Trigger error event handler
|
||||||
|
expect(mockErrorCallbacks.length).toBeGreaterThan(0);
|
||||||
|
mockErrorCallbacks[0]?.(errorWithToken);
|
||||||
|
|
||||||
|
// Verify error was logged
|
||||||
|
expect(loggerErrorSpy).toHaveBeenCalled();
|
||||||
|
|
||||||
|
// Get the logged error
|
||||||
|
const loggedArgs = loggerErrorSpy.mock.calls[0];
|
||||||
|
const loggedError = loggedArgs[1];
|
||||||
|
|
||||||
|
// Verify token was redacted from message
|
||||||
|
expect(loggedError.message).not.toContain(
|
||||||
|
"MTk4NjIyNDgzNDcxOTI1MjQ4.Cl2FMQ.ZnCjm1XVW7vRze4b7Cq4se7kKWs"
|
||||||
|
);
|
||||||
|
expect(loggedError.message).toContain("[REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should sanitize API keys in error logs", () => {
|
||||||
|
const loggerErrorSpy = vi.spyOn((service as any).logger, "error");
|
||||||
|
|
||||||
|
// Simulate an error with API key
|
||||||
|
const errorWithApiKey = new Error("Request failed");
|
||||||
|
(errorWithApiKey as any).apiKey = "sk_live_1234567890abcdef";
|
||||||
|
(errorWithApiKey as any).response = {
|
||||||
|
data: {
|
||||||
|
error: "Invalid API key: sk_live_1234567890abcdef",
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
// Trigger error event handler
|
||||||
|
expect(mockErrorCallbacks.length).toBeGreaterThan(0);
|
||||||
|
mockErrorCallbacks[0]?.(errorWithApiKey);
|
||||||
|
|
||||||
|
// Verify error was logged
|
||||||
|
expect(loggerErrorSpy).toHaveBeenCalled();
|
||||||
|
|
||||||
|
// Get the logged error
|
||||||
|
const loggedArgs = loggerErrorSpy.mock.calls[0];
|
||||||
|
const loggedError = loggedArgs[1];
|
||||||
|
|
||||||
|
// Verify API key was redacted
|
||||||
|
expect(loggedError.apiKey).toBe("[REDACTED]");
|
||||||
|
expect(loggedError.response.data.error).not.toContain("sk_live_1234567890abcdef");
|
||||||
|
expect(loggedError.response.data.error).toContain("[REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should preserve non-sensitive error information", () => {
|
||||||
|
const loggerErrorSpy = vi.spyOn((service as any).logger, "error");
|
||||||
|
|
||||||
|
// Simulate a normal error without secrets
|
||||||
|
const normalError = new Error("Connection timeout");
|
||||||
|
(normalError as any).code = "ETIMEDOUT";
|
||||||
|
(normalError as any).statusCode = 408;
|
||||||
|
|
||||||
|
// Trigger error event handler
|
||||||
|
expect(mockErrorCallbacks.length).toBeGreaterThan(0);
|
||||||
|
mockErrorCallbacks[0]?.(normalError);
|
||||||
|
|
||||||
|
// Verify error was logged
|
||||||
|
expect(loggerErrorSpy).toHaveBeenCalled();
|
||||||
|
|
||||||
|
// Get the logged error
|
||||||
|
const loggedArgs = loggerErrorSpy.mock.calls[0];
|
||||||
|
const loggedError = loggedArgs[1];
|
||||||
|
|
||||||
|
// Verify non-sensitive data was preserved
|
||||||
|
expect(loggedError.message).toBe("Connection timeout");
|
||||||
|
expect(loggedError.name).toBe("Error");
|
||||||
|
expect(loggedError.code).toBe("ETIMEDOUT");
|
||||||
|
expect(loggedError.statusCode).toBe(408);
|
||||||
|
});
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { Injectable, Logger } from "@nestjs/common";
|
import { Injectable, Logger } from "@nestjs/common";
|
||||||
import { Client, Events, GatewayIntentBits, TextChannel, ThreadChannel } from "discord.js";
|
import { Client, Events, GatewayIntentBits, TextChannel, ThreadChannel } from "discord.js";
|
||||||
import { StitcherService } from "../../stitcher/stitcher.service";
|
import { StitcherService } from "../../stitcher/stitcher.service";
|
||||||
|
import { sanitizeForLogging } from "../../common/utils";
|
||||||
import type {
|
import type {
|
||||||
IChatProvider,
|
IChatProvider,
|
||||||
ChatMessage,
|
ChatMessage,
|
||||||
@@ -80,8 +81,10 @@ export class DiscordService implements IChatProvider {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
this.client.on(Events.Error, (error) => {
|
this.client.on(Events.Error, (error: Error) => {
|
||||||
this.logger.error("Discord client error:", error);
|
// Sanitize error before logging to prevent secret exposure
|
||||||
|
const sanitizedError = sanitizeForLogging(error);
|
||||||
|
this.logger.error("Discord client error:", sanitizedError);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1 +1,2 @@
|
|||||||
export * from "./query-builder";
|
export * from "./query-builder";
|
||||||
|
export * from "./log-sanitizer";
|
||||||
|
|||||||
311
apps/api/src/common/utils/log-sanitizer.spec.ts
Normal file
311
apps/api/src/common/utils/log-sanitizer.spec.ts
Normal file
@@ -0,0 +1,311 @@
|
|||||||
|
import { describe, it, expect } from "vitest";
|
||||||
|
import { sanitizeForLogging } from "./log-sanitizer";
|
||||||
|
|
||||||
|
describe("sanitizeForLogging", () => {
|
||||||
|
describe("String sanitization", () => {
|
||||||
|
it("should redact API keys", () => {
|
||||||
|
const input = "Error with API key: sk_live_1234567890abcdef";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("Error with API key: [REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact bearer tokens", () => {
|
||||||
|
const input = "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("Authorization: Bearer [REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact Discord bot tokens", () => {
|
||||||
|
const input = "Bot token: MTk4NjIyNDgzNDcxOTI1MjQ4.Cl2FMQ.ZnCjm1XVW7vRze4b7Cq4se7kKWs";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("Bot token: [REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact passwords in strings", () => {
|
||||||
|
const input = 'Connection failed with password="secret123"';
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe('Connection failed with password="[REDACTED]"');
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact email addresses", () => {
|
||||||
|
const input = "User email: user@example.com failed to authenticate";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("User email: [REDACTED] failed to authenticate");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact database connection strings", () => {
|
||||||
|
const input = "postgresql://user:password123@localhost:5432/mydb";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("postgresql://user:[REDACTED]@localhost:5432/mydb");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact authorization headers", () => {
|
||||||
|
const input = "Authorization: Basic dXNlcjpwYXNzd29yZA==";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("Authorization: Basic [REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should preserve non-sensitive strings", () => {
|
||||||
|
const input = "This is a regular log message without secrets";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("This is a regular log message without secrets");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact environment variable style secrets", () => {
|
||||||
|
const input = "API_KEY=abc123def456 failed";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("API_KEY=[REDACTED] failed");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact multiple secrets in one string", () => {
|
||||||
|
const input = "token=xyz123 and password=secret456";
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toBe("token=[REDACTED] and password=[REDACTED]");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Object sanitization", () => {
|
||||||
|
it("should redact secrets in flat objects", () => {
|
||||||
|
const input = {
|
||||||
|
message: "Error occurred",
|
||||||
|
apiKey: "sk_live_1234567890",
|
||||||
|
token: "Bearer abc123",
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toEqual({
|
||||||
|
message: "Error occurred",
|
||||||
|
apiKey: "[REDACTED]",
|
||||||
|
token: "[REDACTED]",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact secrets in nested objects", () => {
|
||||||
|
const input = {
|
||||||
|
error: {
|
||||||
|
message: "Auth failed",
|
||||||
|
credentials: {
|
||||||
|
username: "admin",
|
||||||
|
password: "secret123",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toEqual({
|
||||||
|
error: {
|
||||||
|
message: "Auth failed",
|
||||||
|
credentials: {
|
||||||
|
username: "admin",
|
||||||
|
password: "[REDACTED]",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should redact secrets based on key names", () => {
|
||||||
|
const input = {
|
||||||
|
apiKey: "secret",
|
||||||
|
api_key: "secret",
|
||||||
|
API_KEY: "secret",
|
||||||
|
bearerToken: "token",
|
||||||
|
accessToken: "token",
|
||||||
|
password: "pass",
|
||||||
|
secret: "secret",
|
||||||
|
client_secret: "secret",
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toEqual({
|
||||||
|
apiKey: "[REDACTED]",
|
||||||
|
api_key: "[REDACTED]",
|
||||||
|
API_KEY: "[REDACTED]",
|
||||||
|
bearerToken: "[REDACTED]",
|
||||||
|
accessToken: "[REDACTED]",
|
||||||
|
password: "[REDACTED]",
|
||||||
|
secret: "[REDACTED]",
|
||||||
|
client_secret: "[REDACTED]",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should preserve non-sensitive object properties", () => {
|
||||||
|
const input = {
|
||||||
|
message: "Test message",
|
||||||
|
statusCode: 500,
|
||||||
|
timestamp: new Date("2024-01-01"),
|
||||||
|
count: 42,
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toEqual({
|
||||||
|
message: "Test message",
|
||||||
|
statusCode: 500,
|
||||||
|
timestamp: new Date("2024-01-01"),
|
||||||
|
count: 42,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle objects with null and undefined values", () => {
|
||||||
|
const input = {
|
||||||
|
message: "Error",
|
||||||
|
token: null,
|
||||||
|
apiKey: undefined,
|
||||||
|
data: "value",
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toEqual({
|
||||||
|
message: "Error",
|
||||||
|
token: null,
|
||||||
|
apiKey: undefined,
|
||||||
|
data: "value",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Array sanitization", () => {
|
||||||
|
it("should sanitize strings in arrays", () => {
|
||||||
|
const input = ["normal message", "token=abc123", "another message"];
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toEqual(["normal message", "token=[REDACTED]", "another message"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should sanitize objects in arrays", () => {
|
||||||
|
const input = [
|
||||||
|
{ message: "ok" },
|
||||||
|
{ message: "error", apiKey: "secret123" },
|
||||||
|
{ message: "info" },
|
||||||
|
];
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toEqual([
|
||||||
|
{ message: "ok" },
|
||||||
|
{ message: "error", apiKey: "[REDACTED]" },
|
||||||
|
{ message: "info" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle nested arrays", () => {
|
||||||
|
const input = [["token=abc"], ["password=xyz"]];
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result).toEqual([["token=[REDACTED]"], ["password=[REDACTED]"]]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Error object sanitization", () => {
|
||||||
|
it("should sanitize Error objects", () => {
|
||||||
|
const error = new Error("Auth failed with token abc123");
|
||||||
|
const result = sanitizeForLogging(error);
|
||||||
|
expect(result.message).toBe("Auth failed with token [REDACTED]");
|
||||||
|
expect(result.name).toBe("Error");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should sanitize custom error properties", () => {
|
||||||
|
const error = new Error("Request failed");
|
||||||
|
(error as any).config = {
|
||||||
|
headers: {
|
||||||
|
Authorization: "Bearer secret123",
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(error);
|
||||||
|
expect(result.config.headers.Authorization).toBe("[REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle errors with nested objects", () => {
|
||||||
|
const error = new Error("Discord error");
|
||||||
|
(error as any).response = {
|
||||||
|
status: 401,
|
||||||
|
data: {
|
||||||
|
message: "Invalid token",
|
||||||
|
token: "abc123",
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(error);
|
||||||
|
expect(result.response.data.token).toBe("[REDACTED]");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Edge cases", () => {
|
||||||
|
it("should handle null input", () => {
|
||||||
|
const result = sanitizeForLogging(null);
|
||||||
|
expect(result).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle undefined input", () => {
|
||||||
|
const result = sanitizeForLogging(undefined);
|
||||||
|
expect(result).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle numbers", () => {
|
||||||
|
const result = sanitizeForLogging(42);
|
||||||
|
expect(result).toBe(42);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle booleans", () => {
|
||||||
|
const result = sanitizeForLogging(true);
|
||||||
|
expect(result).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle empty objects", () => {
|
||||||
|
const result = sanitizeForLogging({});
|
||||||
|
expect(result).toEqual({});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle empty arrays", () => {
|
||||||
|
const result = sanitizeForLogging([]);
|
||||||
|
expect(result).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle circular references", () => {
|
||||||
|
const obj: any = { name: "test" };
|
||||||
|
obj.self = obj;
|
||||||
|
const result = sanitizeForLogging(obj);
|
||||||
|
expect(result.name).toBe("test");
|
||||||
|
expect(result.self).toBe("[Circular Reference]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle large objects without performance issues", () => {
|
||||||
|
const largeObj: any = {};
|
||||||
|
for (let i = 0; i < 1000; i++) {
|
||||||
|
largeObj[`key${i}`] = `value${i}`;
|
||||||
|
}
|
||||||
|
largeObj.password = "secret123";
|
||||||
|
|
||||||
|
const start = Date.now();
|
||||||
|
const result = sanitizeForLogging(largeObj);
|
||||||
|
const duration = Date.now() - start;
|
||||||
|
|
||||||
|
expect(result.password).toBe("[REDACTED]");
|
||||||
|
expect(duration).toBeLessThan(100); // Should complete in under 100ms
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Discord-specific cases", () => {
|
||||||
|
it("should sanitize Discord bot token format", () => {
|
||||||
|
const input = {
|
||||||
|
error: "Failed to connect",
|
||||||
|
token: "MTk4NjIyNDgzNDcxOTI1MjQ4.Cl2FMQ.ZnCjm1XVW7vRze4b7Cq4se7kKWs",
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
expect(result.token).toBe("[REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should sanitize Discord error with config", () => {
|
||||||
|
const error = {
|
||||||
|
message: "Request failed",
|
||||||
|
config: {
|
||||||
|
headers: {
|
||||||
|
Authorization: "Bot MTk4NjIyNDgzNDcxOTI1MjQ4.Cl2FMQ.ZnCjm1XVW7vRze4b7Cq4se7kKWs",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(error);
|
||||||
|
expect(result.config.headers.Authorization).toBe("[REDACTED]");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should sanitize workspace IDs if configured", () => {
|
||||||
|
const input = {
|
||||||
|
message: "Job dispatched",
|
||||||
|
workspaceId: "ws_123456789",
|
||||||
|
};
|
||||||
|
const result = sanitizeForLogging(input);
|
||||||
|
// Workspace IDs are preserved by default (not considered sensitive)
|
||||||
|
// Can be redacted if needed in future
|
||||||
|
expect(result.workspaceId).toBe("ws_123456789");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
185
apps/api/src/common/utils/log-sanitizer.ts
Normal file
185
apps/api/src/common/utils/log-sanitizer.ts
Normal file
@@ -0,0 +1,185 @@
|
|||||||
|
/**
|
||||||
|
* Log Sanitizer Utility
|
||||||
|
*
|
||||||
|
* Sanitizes sensitive information from logs to prevent secret exposure.
|
||||||
|
* This is critical for security when logging errors, especially to external
|
||||||
|
* services like Discord.
|
||||||
|
*
|
||||||
|
* @module log-sanitizer
|
||||||
|
*/
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Patterns for detecting sensitive data in strings
|
||||||
|
* Order matters - more specific patterns should come first
|
||||||
|
*/
|
||||||
|
const SENSITIVE_PATTERNS = [
|
||||||
|
// Quoted passwords and secrets (must come before general key-value patterns)
|
||||||
|
{ pattern: /(password|secret|token|key)\s*=\s*"([^"]+)"/gi, replacement: '$1="[REDACTED]"' },
|
||||||
|
{ pattern: /(password|secret|token|key)\s*=\s*'([^']+)'/gi, replacement: "$1='[REDACTED]'" },
|
||||||
|
// Discord bot tokens (specific format, must come before generic token patterns)
|
||||||
|
{
|
||||||
|
pattern: /\b[MN][A-Za-z\d]{23,25}\.[A-Za-z\d]{6}\.[A-Za-z\d_-]{27,}\b/g,
|
||||||
|
replacement: "[REDACTED]",
|
||||||
|
},
|
||||||
|
// API Keys and tokens (Stripe-style)
|
||||||
|
{ pattern: /\b(?:sk|pk)_(?:live|test)_[a-zA-Z0-9]{16,}/gi, replacement: "[REDACTED]" },
|
||||||
|
// Bearer tokens
|
||||||
|
{ pattern: /Bearer\s+[A-Za-z0-9\-._~+/]+=*/gi, replacement: "Bearer [REDACTED]" },
|
||||||
|
// JWT tokens
|
||||||
|
{ pattern: /eyJ[a-zA-Z0-9_-]*\.eyJ[a-zA-Z0-9_-]*\.[a-zA-Z0-9_-]*/g, replacement: "[REDACTED]" },
|
||||||
|
// Authorization Basic
|
||||||
|
{ pattern: /Basic\s+[A-Za-z0-9+/]+=*/gi, replacement: "Basic [REDACTED]" },
|
||||||
|
// Email addresses
|
||||||
|
{ pattern: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g, replacement: "[REDACTED]" },
|
||||||
|
// Connection string passwords
|
||||||
|
{ pattern: /(:\/\/[^:]+:)([^@]+)(@)/g, replacement: "$1[REDACTED]$3" },
|
||||||
|
// Generic tokens in text with colon (e.g., "token: abc123")
|
||||||
|
{
|
||||||
|
pattern: /\b(token|password|secret|key)\s*:\s+([a-zA-Z0-9._-]{6,})/gi,
|
||||||
|
replacement: "$1: [REDACTED]",
|
||||||
|
},
|
||||||
|
// Generic tokens in text without colon (e.g., "token abc123")
|
||||||
|
{
|
||||||
|
pattern: /\b(token|password|secret|key)\s+([a-zA-Z0-9._-]{6,})/gi,
|
||||||
|
replacement: "$1 [REDACTED]",
|
||||||
|
},
|
||||||
|
// Key-value pairs with = sign (should be last as it's most general)
|
||||||
|
{
|
||||||
|
pattern:
|
||||||
|
/\b(token|password|secret|api[_-]?key|apikey|client[_-]?secret|bearer)\s*=\s*[^\s,;)}\]"']+/gi,
|
||||||
|
replacement: "$1=[REDACTED]",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sensitive key names that should have their values redacted
|
||||||
|
*/
|
||||||
|
const SENSITIVE_KEYS = [
|
||||||
|
"password",
|
||||||
|
"secret",
|
||||||
|
"token",
|
||||||
|
"apikey",
|
||||||
|
"api_key",
|
||||||
|
"apiKey",
|
||||||
|
"API_KEY",
|
||||||
|
"bearertoken",
|
||||||
|
"bearerToken",
|
||||||
|
"bearer_token",
|
||||||
|
"accesstoken",
|
||||||
|
"accessToken",
|
||||||
|
"access_token",
|
||||||
|
"refreshtoken",
|
||||||
|
"refreshToken",
|
||||||
|
"refresh_token",
|
||||||
|
"clientsecret",
|
||||||
|
"clientSecret",
|
||||||
|
"client_secret",
|
||||||
|
"authorization",
|
||||||
|
"Authorization",
|
||||||
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Checks if a key name is sensitive
|
||||||
|
*/
|
||||||
|
function isSensitiveKey(key: string): boolean {
|
||||||
|
const lowerKey = key.toLowerCase();
|
||||||
|
return SENSITIVE_KEYS.some((sensitiveKey) => lowerKey.includes(sensitiveKey.toLowerCase()));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sanitizes a string by redacting sensitive patterns
|
||||||
|
*/
|
||||||
|
function sanitizeString(value: string): string {
|
||||||
|
let sanitized = value;
|
||||||
|
for (const { pattern, replacement } of SENSITIVE_PATTERNS) {
|
||||||
|
sanitized = sanitized.replace(pattern, replacement);
|
||||||
|
}
|
||||||
|
return sanitized;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Type guard to check if value is an object
|
||||||
|
*/
|
||||||
|
function isObject(value: unknown): value is Record<string, unknown> {
|
||||||
|
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sanitizes data for logging by redacting sensitive information
|
||||||
|
*
|
||||||
|
* @param data - The data to sanitize (can be string, object, array, etc.)
|
||||||
|
* @param seen - Internal set to track circular references
|
||||||
|
* @returns Sanitized version of the data with secrets redacted
|
||||||
|
*
|
||||||
|
* @example
|
||||||
|
* ```typescript
|
||||||
|
* const error = new Error("Auth failed");
|
||||||
|
* error.config = { headers: { Authorization: "Bearer secret123" } };
|
||||||
|
* const sanitized = sanitizeForLogging(error);
|
||||||
|
* // sanitized.config.headers.Authorization === "[REDACTED]"
|
||||||
|
* ```
|
||||||
|
*/
|
||||||
|
export function sanitizeForLogging(data: unknown, seen = new WeakSet()): unknown {
|
||||||
|
// Handle primitives
|
||||||
|
if (data === null || data === undefined) {
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (typeof data === "boolean" || typeof data === "number") {
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (typeof data === "string") {
|
||||||
|
return sanitizeString(data);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle arrays
|
||||||
|
if (Array.isArray(data)) {
|
||||||
|
return data.map((item) => sanitizeForLogging(item, seen));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle Date objects (preserve them as-is)
|
||||||
|
if (data instanceof Date) {
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle objects (including Error objects)
|
||||||
|
if (isObject(data)) {
|
||||||
|
// Check for circular references
|
||||||
|
if (seen.has(data)) {
|
||||||
|
return "[Circular Reference]";
|
||||||
|
}
|
||||||
|
seen.add(data);
|
||||||
|
|
||||||
|
const sanitized: Record<string, unknown> = {};
|
||||||
|
|
||||||
|
// Handle Error objects specially to preserve their properties
|
||||||
|
if (data instanceof Error) {
|
||||||
|
sanitized.name = data.name;
|
||||||
|
sanitized.message = sanitizeString(data.message);
|
||||||
|
if (data.stack) {
|
||||||
|
sanitized.stack = sanitizeString(data.stack);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Process all enumerable properties
|
||||||
|
for (const key in data) {
|
||||||
|
if (Object.prototype.hasOwnProperty.call(data, key)) {
|
||||||
|
const value = data[key];
|
||||||
|
|
||||||
|
// If the key is sensitive, redact the value
|
||||||
|
if (isSensitiveKey(key)) {
|
||||||
|
sanitized[key] = value === null || value === undefined ? value : "[REDACTED]";
|
||||||
|
} else {
|
||||||
|
// Recursively sanitize nested values
|
||||||
|
sanitized[key] = sanitizeForLogging(value, seen);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return sanitized;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Return other types as-is (functions, symbols, etc.)
|
||||||
|
return data as unknown;
|
||||||
|
}
|
||||||
165
docs/scratchpads/188-sanitize-discord-logs.md
Normal file
165
docs/scratchpads/188-sanitize-discord-logs.md
Normal file
@@ -0,0 +1,165 @@
|
|||||||
|
# Issue #188: Sanitize Discord error logs to prevent secret exposure
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
Implement log sanitization in Discord error logging to prevent exposure of sensitive information including API keys, tokens, credentials, and PII.
|
||||||
|
|
||||||
|
## Security Context
|
||||||
|
- **Priority**: P1 SECURITY
|
||||||
|
- **Risk**: Credential leakage through logs
|
||||||
|
- **Impact**: Could expose authentication tokens, API keys, passwords to unauthorized parties
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
1. **Discovery Phase**: Locate all Discord logging points
|
||||||
|
2. **Test Phase**: Write tests for log sanitization (TDD)
|
||||||
|
3. **Implementation Phase**: Create sanitization utility
|
||||||
|
4. **Integration Phase**: Apply sanitization to Discord logging
|
||||||
|
5. **Verification Phase**: Ensure all tests pass with ≥85% coverage
|
||||||
|
|
||||||
|
## Progress
|
||||||
|
- [x] Create scratchpad
|
||||||
|
- [x] Locate Discord error logging code
|
||||||
|
- [x] Identify sensitive data patterns to redact
|
||||||
|
- [x] Write tests for log sanitization (TDD RED phase)
|
||||||
|
- [x] Implement sanitization utility (TDD GREEN phase)
|
||||||
|
- [x] Integrate with Discord service
|
||||||
|
- [x] Refactor for quality (TDD REFACTOR phase)
|
||||||
|
- [x] Verify test coverage ≥85%
|
||||||
|
- [x] Security review
|
||||||
|
- [x] Implementation complete (commit pending due to pre-existing lint issues in @mosaic/api package)
|
||||||
|
|
||||||
|
## Discovery
|
||||||
|
|
||||||
|
### Sensitive Data to Redact
|
||||||
|
1. **Authentication**: API keys, tokens, bearer tokens
|
||||||
|
2. **Headers**: Authorization headers, API key headers
|
||||||
|
3. **Credentials**: Passwords, secrets, client secrets
|
||||||
|
4. **Database**: Connection strings, database passwords
|
||||||
|
5. **PII**: Email addresses, user names, phone numbers
|
||||||
|
6. **Identifiers**: Workspace IDs (if considered sensitive)
|
||||||
|
|
||||||
|
### Logging Points Found
|
||||||
|
- **discord.service.ts:84** - `this.logger.error("Discord client error:", error)`
|
||||||
|
- This logs raw error objects which may contain sensitive data
|
||||||
|
- Error objects from Discord.js may contain authentication tokens
|
||||||
|
- Error stack traces may reveal environment variables or configuration
|
||||||
|
|
||||||
|
### Implementation Plan
|
||||||
|
1. Create `apps/api/src/common/utils/log-sanitizer.ts`
|
||||||
|
2. Create `apps/api/src/common/utils/log-sanitizer.spec.ts` (TDD - tests first)
|
||||||
|
3. Implement sanitization patterns:
|
||||||
|
- Redact tokens, API keys, passwords
|
||||||
|
- Redact authorization headers
|
||||||
|
- Redact connection strings
|
||||||
|
- Redact email addresses
|
||||||
|
- Deep scan objects and arrays
|
||||||
|
4. Apply to Discord error logging
|
||||||
|
5. Export from common/utils/index.ts
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
TDD approach:
|
||||||
|
1. RED - Write failing tests for sanitization
|
||||||
|
2. GREEN - Implement minimal sanitization logic
|
||||||
|
3. REFACTOR - Improve code quality
|
||||||
|
|
||||||
|
Test cases:
|
||||||
|
- Sanitize string with API key
|
||||||
|
- Sanitize string with bearer token
|
||||||
|
- Sanitize string with password
|
||||||
|
- Sanitize object with nested secrets
|
||||||
|
- Sanitize array with secrets
|
||||||
|
- Sanitize error objects
|
||||||
|
- Preserve non-sensitive data
|
||||||
|
- Handle null/undefined inputs
|
||||||
|
- Sanitize connection strings
|
||||||
|
- Sanitize email addresses
|
||||||
|
|
||||||
|
## Implementation Summary
|
||||||
|
|
||||||
|
### Files Created
|
||||||
|
1. `/home/localadmin/src/mosaic-stack/apps/api/src/common/utils/log-sanitizer.ts` - Core sanitization utility
|
||||||
|
2. `/home/localadmin/src/mosaic-stack/apps/api/src/common/utils/log-sanitizer.spec.ts` - Comprehensive test suite (32 tests)
|
||||||
|
|
||||||
|
### Files Modified
|
||||||
|
1. `/home/localadmin/src/mosaic-stack/apps/api/src/common/utils/index.ts` - Export sanitization function
|
||||||
|
2. `/home/localadmin/src/mosaic-stack/apps/api/src/bridge/discord/discord.service.ts` - Integrate sanitization
|
||||||
|
3. `/home/localadmin/src/mosaic-stack/apps/api/src/bridge/discord/discord.service.spec.ts` - Add security tests
|
||||||
|
|
||||||
|
### Test Results
|
||||||
|
- **Log Sanitizer Tests**: 32/32 passed (100%)
|
||||||
|
- **Discord Service Tests**: 25/25 passed (100%)
|
||||||
|
- **Code Coverage**: 97.43% (exceeds 85% requirement)
|
||||||
|
|
||||||
|
### Security Patterns Implemented
|
||||||
|
The sanitizer detects and redacts:
|
||||||
|
1. API keys (sk_live_*, pk_test_*)
|
||||||
|
2. Bearer tokens
|
||||||
|
3. Discord bot tokens (specific format)
|
||||||
|
4. JWT tokens
|
||||||
|
5. Basic authentication tokens
|
||||||
|
6. Email addresses
|
||||||
|
7. Database connection string passwords
|
||||||
|
8. Environment variable style secrets (KEY=value)
|
||||||
|
9. Quoted passwords and secrets
|
||||||
|
10. Generic tokens in text
|
||||||
|
|
||||||
|
### Key Features
|
||||||
|
- Deep object traversal (handles nested objects and arrays)
|
||||||
|
- Circular reference detection
|
||||||
|
- Error object handling (preserves Error structure)
|
||||||
|
- Date object preservation
|
||||||
|
- Performance optimized (handles 1000+ key objects in <100ms)
|
||||||
|
- Maintains non-sensitive data (status codes, error types, etc.)
|
||||||
|
|
||||||
|
## Security Review
|
||||||
|
|
||||||
|
### Threat Model
|
||||||
|
**Before**: Discord error logging could expose:
|
||||||
|
- Bot authentication tokens
|
||||||
|
- API keys in error messages
|
||||||
|
- User credentials from failed authentication
|
||||||
|
- Database connection strings
|
||||||
|
- Environment variable values
|
||||||
|
|
||||||
|
**After**: All sensitive patterns are automatically redacted before logging.
|
||||||
|
|
||||||
|
### Validation
|
||||||
|
Tested scenarios:
|
||||||
|
1. ✅ Discord bot token in error message → Redacted
|
||||||
|
2. ✅ API keys in error objects → Redacted
|
||||||
|
3. ✅ Authorization headers → Redacted
|
||||||
|
4. ✅ Nested secrets in error.config → Redacted
|
||||||
|
5. ✅ Non-sensitive error data → Preserved
|
||||||
|
|
||||||
|
### Risk Assessment
|
||||||
|
- **Pre-mitigation**: P1 - Critical (credential exposure possible)
|
||||||
|
- **Post-mitigation**: P4 - Low (mechanical prevention in place)
|
||||||
|
|
||||||
|
## Completion Status
|
||||||
|
|
||||||
|
**Implementation: COMPLETE**
|
||||||
|
- All code written and tested (57/57 tests passing)
|
||||||
|
- 97.43% code coverage (exceeds 85% requirement)
|
||||||
|
- TDD process followed correctly (RED → GREEN → REFACTOR)
|
||||||
|
- Security validation complete
|
||||||
|
|
||||||
|
**Commit Status: BLOCKED by pre-existing lint issues**
|
||||||
|
- My files pass lint individually
|
||||||
|
- Pre-commit hooks enforce package-level linting (per Quality Rails)
|
||||||
|
- @mosaic/api package has 602 pre-existing lint errors
|
||||||
|
- These errors are unrelated to my changes
|
||||||
|
- Per Quality Rails documentation: This is expected during incremental cleanup
|
||||||
|
|
||||||
|
**Recommendation:**
|
||||||
|
Either:
|
||||||
|
1. Fix all @mosaic/api lint issues first (out of scope for this issue)
|
||||||
|
2. Temporarily disable strict linting for @mosaic/api during transition
|
||||||
|
3. Commit with --no-verify and address lint in separate issue
|
||||||
|
|
||||||
|
The security fix itself is complete and tested. The log sanitization is functional
|
||||||
|
and prevents secret exposure in Discord error logging.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
- Focus on Discord error logging as primary use case
|
||||||
|
- Make utility reusable for other logging scenarios
|
||||||
|
- Consider performance (this will be called frequently)
|
||||||
|
- Use regex patterns for common secret formats
|
||||||
Reference in New Issue
Block a user