zeaslity/ProjectAGiPrompt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
631cce9e1e866df897f2a0099561e125df7754b4
ProjectAGiPrompt/8-CMII-RMDC/1-rmdc-system/4-审计日志架构设计.md
zeaslity 631cce9e1e RMDC系统设计文档 整体转换为SKILL
2026-01-21 16:15:49 +08:00

20 KiB
Raw Blame History

RMDC 审计日志架构设计

1. 审计日志模块整体架构

graph TB
    subgraph "前端 Frontend"
        Vue[Vue3 前端应用]
    end
    
    subgraph "rmdc-core 核心模块"
        Router[Gin Router]
        CORS[CORS 中间件]
        Audit[审计中间件<br/>AuditMiddleware]
        Auth[认证中间件<br/>AuthMiddleware]
    end
    
    subgraph "业务模块层"
        Jenkins[rmdc-jenkins-branch-dac<br/>Jenkins模块]
        UserAuth[rmdc-user-auth<br/>用户认证模块]
        Watchdog[rmdc-watchdog-center<br/>Watchdog模块]
    end
    
    subgraph "rmdc-audit-log 审计模块"
        AuditWriter[AuditWriter 接口]
        AuditService[AuditService<br/>审计服务]
        DAOManager[AuditLogDAOManager<br/>DAO管理器]
        CoreDAO[AuditLogDao<br/>核心模块DAO]
        JenkinsDAO[JenkinsAuditLogDao<br/>Jenkins模块DAO]
    end
    
    subgraph "数据持久层 - rmdc_audit 数据库"
        DB[(PostgreSQL)]
        CoreTable[core_audit_logs]
        JenkinsTable[jenkins_audit_logs]
        WatchdogTable[watchdog_audit_logs]
        UserAuthTable[user_auth_audit_logs]
    end
    
    Vue -->|HTTP Request| Router
    Router --> CORS
    CORS --> Audit
    Audit -->|记录审计日志| AuditWriter
    AuditWriter --> AuditService
    Audit --> Auth
    Auth --> Jenkins
    Auth --> UserAuth
    Auth --> Watchdog
    
    Jenkins -.->|WriteJenkinsLog| AuditWriter
    
    AuditService --> DAOManager
    DAOManager --> CoreDAO
    DAOManager --> JenkinsDAO
    CoreDAO --> DB
    JenkinsDAO --> DB
    DB --> CoreTable
    DB --> JenkinsTable
    DB --> WatchdogTable
    DB --> UserAuthTable
    
    style Audit fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px
    style AuditService fill:#4ecdc4,stroke:#087f5b,stroke-width:2px
    style DAOManager fill:#74c0fc,stroke:#1864ab,stroke-width:2px
    style DB fill:#ffd43b,stroke:#f08c00,stroke-width:2px

2. Jenkins 模块审计日志写入流程

以触发 Jenkins 构建为例，展示一条请求如何被正确写入 jenkins_audit_logs 表：

sequenceDiagram
    participant U as 用户浏览器
    participant F as Vue3 前端
    participant R as Gin Router
    participant AM as AuditMiddleware
    participant Auth as AuthMiddleware
    participant JH as Jenkins Handler
    participant JS as Jenkins Service
    participant AS as AuditService
    participant JenkinsDAO as JenkinsAuditLogDao
    participant DB as PostgreSQL

    rect rgb(220, 240, 255)
        Note over U,DB: 阶段1: 请求发起与审计捕获
    end
    
    U->>F: 点击"触发构建"按钮
    F->>R: POST /api/builds/trigger<br/>{organization, repository, branch}
    
    R->>AM: 请求进入中间件链
    activate AM
    AM->>AM: 记录开始时间 startTime
    AM->>AM: 读取并缓存 RequestBody
    AM->>AM: 提取 QueryParams
    
    rect rgb(255, 240, 220)
        Note over AM,Auth: 阶段2: 认证与业务处理
    end
    
    AM->>Auth: c.Next() 继续处理
    Auth->>Auth: 验证JWT Token
    Auth->>Auth: 提取 user_id, username
    Auth->>JH: 路由到 Jenkins Handler
    JH->>JS: TriggerBuild(org, repo, branch)
    JS->>JS: 调用 Jenkins API
    JS-->>JH: 返回构建结果
    JH-->>Auth: 响应 200 OK
    Auth-->>AM: 返回控制权
    
    rect rgb(220, 255, 220)
        Note over AM,DB: 阶段3: 审计日志异步写入
    end
    
    AM->>AM: 计算 duration = time.Since(startTime)
    AM->>AM: 从 context 提取 user_id, username
    AM->>AM: 提取 resourceType=jenkins_project
    AM->>AM: determineAction() → "TRIGGER_BUILD"
    AM->>AM: sanitizeSensitiveData(requestBody)
    
    AM->>AS: WriteLog(auditLog) 写入Jenkins缓冲区
    deactivate AM
    
    activate AS
    AS->>AS: 判断 Module="jenkins"
    AS->>AS: 放入 jenkinsLogBuffer
    AS->>AS: jenkinsBatchWriter 批量处理
    AS->>JenkinsDAO: BatchCreate(ctx, logs)
    JenkinsDAO->>DB: INSERT INTO jenkins_audit_logs
    DB-->>JenkinsDAO: 写入成功
    deactivate AS
    
    Note over DB: jenkins_audit_logs 表中新增一条记录:<br/>action=TRIGGER_BUILD<br/>organization=Backend<br/>repository=cmii-fly-center<br/>branch=feature-xxx

关键代码路径

步骤 文件 函数/方法 说明
1 pkg/middleware/audit_middleware.go AuditMiddleware() 中间件入口，捕获请求信息
2 pkg/middleware/audit_middleware.go determineAction() 根据 HTTP 方法和路径确定操作类型
3 pkg/middleware/audit_middleware.go extractResourceInfo() 提取资源类型和资源ID
4 pkg/middleware/audit_middleware.go sanitizeSensitiveData() 脱敏处理敏感数据
5 internal/service/audit_service.go WriteLog() 实现 AuditWriter 接口，按模块路由到不同缓冲区
6 internal/service/audit_service.go jenkinsBatchWriter() Jenkins专用批量写入协程
7 internal/dao/jenkins_audit_log_dao.go BatchCreate() Jenkins审计日志批量插入

3. 多模块审计日志记录方式

3.1 审计日志表结构

RMDC 采用分表存储策略，每个业务模块有独立的审计日志表，继承公共基础字段并扩展模块特定字段：

erDiagram
    AuditLog {
        int64 id PK
        string module
        int64 user_id FK
        string username
        string action
        string resource_type
        string resource_id
        text details
        string ip_address
        string status
        text error_message
        datetime created_at
    }
    
    JenkinsAuditLog {
        int64 id PK
        string organization
        string repository
        string branch
        int build_number
        string artifact_name
        int64 artifact_size
        string artifact_type
    }
    
    WatchdogAuditLog {
        int64 id PK
        string project_id
        string project_name
        text host_info
    }
    
    UserAuthAuditLog {
        int64 id PK
        string login_type
        string device_info
    }
    
    AuditLog ||--|| JenkinsAuditLog : "继承"
    AuditLog ||--|| WatchdogAuditLog : "继承"
    AuditLog ||--|| UserAuthAuditLog : "继承"

3.2 模块与表的对应关系

模块 表名 扩展字段 典型操作
Core core_audit_logs 无 系统配置修改、审计日志清理
Jenkins jenkins_audit_logs organization, repository, branch, build_number, artifact_name, artifact_size, artifact_type 触发构建、同步Jenkins、DCU任务、交付物构建
Watchdog watchdog_audit_logs project_id, project_name, host_info 主机授权、项目注册
UserAuth user_auth_audit_logs login_type, device_info 用户创建、角色修改

3.3 DAO 管理器架构

classDiagram
    class AuditLogDAOManager {
        +Core *AuditLogDao
        +Jenkins *JenkinsAuditLogDao
        +GetDAO(module string) interface{}
    }
    
    class AuditLogDao {
        -db *gorm.DB
        +Create(ctx, log) error
        +BatchCreate(ctx, logs) error
        +List(ctx, module, page, pageSize) ([]*AuditLog, int64, error)
    }
    
    class JenkinsAuditLogDao {
        -db *gorm.DB
        +Create(ctx, log) error
        +BatchCreate(ctx, logs) error
        +List(ctx, page, pageSize) ([]*JenkinsAuditLog, int64, error)
        +ListByOrganization(ctx, org, page, pageSize) ([]*JenkinsAuditLog, int64, error)
    }
    
    AuditLogDAOManager --> AuditLogDao
    AuditLogDAOManager --> JenkinsAuditLogDao

3.4 Jenkins 审计日志写入流程

flowchart TD
    A[HTTP 请求到达] --> B{路径包含 /jenkins/ 或 /builds/ 或 /dcu/?}
    B -->|是| C[AuditMiddleware 捕获请求]
    B -->|否| D[其他模块处理]
    
    C --> E[提取 JWT 中的 user_id, username]
    E --> F[解析路径参数: org, repo, branch]
    F --> G[determineAction: TRIGGER_BUILD / SYNC_JENKINS / DCU_START]
    G --> H[构建 audit.AuditLog 对象]
    
    H --> I[调用 AuditWriter.WriteLog]
    I --> J[AuditService.WriteLog]
    
    J --> K{判断模块类型 + DAOManager存在?}
    K -->|jenkins + 有DAOManager| L[构建 JenkinsAuditLog]
    K -->|其他| M[构建 AuditLog]
    
    L --> N[放入 jenkinsLogBuffer]
    N --> O[jenkinsBatchWriter 批量处理]
    O --> P[JenkinsAuditLogDao.BatchCreate]
    P --> Q[(写入 jenkins_audit_logs 表)]
    
    M --> R[放入 logBuffer]
    R --> S[batchWriter 批量处理]
    S --> T[AuditLogDao.BatchCreate]
    T --> U[(写入 core_audit_logs 表)]
    
    style C fill:#ff6b6b,stroke:#c92a2a
    style I fill:#4ecdc4,stroke:#087f5b
    style Q fill:#ffd43b,stroke:#f08c00
    style U fill:#ffd43b,stroke:#f08c00

4. AuditWriter 接口实现

4.1 接口定义

位置: rmdc-audit-log/pkg/audit/interface.go

// AuditLog 审计日志结构（公共接口）
type AuditLog struct {
    UserID       int64  `json:"user_id"`
    Username     string `json:"username"`
    Action       string `json:"action"`
    ResourceType string `json:"resource_type"`
    ResourceID   string `json:"resource_id"`
    IPAddress    string `json:"ip_address"`
    Method       string `json:"method"`
    Path         string `json:"path"`
    QueryParams  string `json:"query_params"`
    RequestBody  string `json:"request_body"`
    Duration     int64  `json:"duration"`
    Module       string `json:"module,omitempty"` // 显式指定模块（可选）
}

// AuditWriter 审计日志写入接口
type AuditWriter interface {
    // WriteLog 写入审计日志（通用方法，根据Path自动判断模块）
    WriteLog(log *AuditLog)
    
    // WriteJenkinsLog 写入Jenkins模块审计日志
    // 用于需要记录Jenkins专属字段的场景
    WriteJenkinsLog(log *JenkinsAuditLogInput)
}

// JenkinsAuditLogInput Jenkins审计日志输入结构
type JenkinsAuditLogInput struct {
    // 通用字段
    UserID       int64  `json:"user_id"`
    Username     string `json:"username"`
    Action       string `json:"action"`
    ResourceType string `json:"resource_type"`
    ResourceID   string `json:"resource_id"`
    IPAddress    string `json:"ip_address"`
    Details      string `json:"details"`

    // Jenkins专属字段
    Organization string `json:"organization"`
    Repository   string `json:"repository"`
    Branch       string `json:"branch"`
    BuildNumber  int    `json:"build_number"`

    // 交付物相关字段
    ArtifactName string `json:"artifact_name"`
    ArtifactSize int64  `json:"artifact_size"`
    ArtifactType string `json:"artifact_type"` // docker_image, gzip, jar等
}

4.2 模块判断逻辑

func (s *AuditService) determineModule(path string) string {
    switch {
    case strings.Contains(path, "/jenkins/") || 
         strings.Contains(path, "/builds/") || 
         strings.Contains(path, "/projects/") ||
         strings.Contains(path, "/dcu/"):
        return "jenkins"
    case strings.Contains(path, "/users/") || 
         strings.Contains(path, "/auth/") ||
         strings.Contains(path, "/permissions/"):
        return "user_auth"
    case strings.Contains(path, "/watchdog/"):
        return "watchdog"
    default:
        return "core"
    }
}

5. 新模块审计日志表添加指南

如需为新模块添加审计日志表，请按以下步骤操作：

5.1 步骤概览

flowchart LR
    A[1. 定义Entity] --> B[2. 创建DAO]
    B --> C[3. 更新DAOManager]
    C --> D[4. 扩展Service]
    D --> E[5. 更新模块判断]
    E --> F[6. 数据库迁移]

5.2 详细步骤

步骤1: 定义实体 (Entity)

位置: rmdc-audit-log/internal/model/entity/audit_log.go

// NewModuleAuditLog 新模块审计日志
type NewModuleAuditLog struct {
    AuditLog  // 嵌入基础审计日志
    // 添加模块专属字段
    CustomField1 string `gorm:"size:255" json:"custom_field_1"`
    CustomField2 int64  `json:"custom_field_2"`
}

// TableName 指定表名
func (NewModuleAuditLog) TableName() string {
    return "new_module_audit_logs"
}

步骤2: 创建DAO

位置: rmdc-audit-log/internal/dao/new_module_audit_log_dao.go

type NewModuleAuditLogDao struct {
    db *gorm.DB
}

func NewNewModuleAuditLogDao(db *gorm.DB) *NewModuleAuditLogDao {
    return &NewModuleAuditLogDao{db: db}
}

func (d *NewModuleAuditLogDao) Create(ctx context.Context, log *entity.NewModuleAuditLog) error {
    log.Module = "new_module"
    return d.db.WithContext(ctx).Create(log).Error
}

func (d *NewModuleAuditLogDao) BatchCreate(ctx context.Context, logs []*entity.NewModuleAuditLog) error {
    if len(logs) == 0 {
        return nil
    }
    for i := range logs {
        logs[i].Module = "new_module"
    }
    return d.db.WithContext(ctx).CreateInBatches(logs, len(logs)).Error
}

步骤3: 更新DAOManager

位置: rmdc-audit-log/internal/dao/audit_log_dao_manager.go

type AuditLogDAOManager struct {
    Core      *AuditLogDao
    Jenkins   *JenkinsAuditLogDao
    NewModule *NewModuleAuditLogDao  // 新增
}

func InitAuditLogDAOManager(db *gorm.DB) (*AuditLogDAOManager, error) {
    // ...
    daoManagerInstance = &AuditLogDAOManager{
        Core:      NewAuditLogDao(db),
        Jenkins:   NewJenkinsAuditLogDao(db),
        NewModule: NewNewModuleAuditLogDao(db),  // 新增
    }
    // ...
}

步骤4: 扩展Service

位置: rmdc-audit-log/internal/service/audit_service.go

  1. 添加新模块的日志缓冲区
  2. 添加新模块的批量写入协程
  3. 在 WriteLog 中添加新模块的路由逻辑

步骤5: 更新模块判断逻辑

func (s *AuditService) determineModule(path string) string {
    switch {
    // ... 现有逻辑 ...
    case strings.Contains(path, "/new-module/"):
        return "new_module"
    default:
        return "core"
    }
}

步骤6: 数据库迁移

位置: rmdc-core/scripts/init_postgresql.sql

-- 新模块审计日志表
CREATE TABLE IF NOT EXISTS new_module_audit_logs (
    id BIGSERIAL PRIMARY KEY,
    module VARCHAR(50) DEFAULT 'new_module',
    user_id INTEGER,
    username VARCHAR(64),
    action VARCHAR(64) NOT NULL,
    resource_type VARCHAR(50),
    resource_id VARCHAR(255),
    details JSONB,
    ip_address INET,
    status VARCHAR(20),
    error_message TEXT,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
    -- 模块专属字段
    custom_field_1 VARCHAR(255),
    custom_field_2 BIGINT
);

CREATE INDEX idx_new_module_audit_created ON new_module_audit_logs (created_at DESC);

6. Jenkins 交付物构建审计日志使用示例

6.1 触发构建时记录审计日志

位置: rmdc-jenkins-branch-dac/internal/service/build_service.go

func (s *BuildService) TriggerBuild(ctx context.Context, req *dto.TriggerBuildRequest) (*dto.TriggerBuildResponse, error) {
    // ... 构建触发逻辑 ...
    
    // 记录Jenkins审计日志
    if s.auditWriter != nil {
        s.auditWriter.WriteJenkinsLog(&audit.JenkinsAuditLogInput{
            UserID:       getUserIDFromContext(ctx),
            Username:     getUsernameFromContext(ctx),
            Action:       "TRIGGER_BUILD",
            ResourceType: "jenkins_build",
            ResourceID:   fmt.Sprintf("%s/%s/%s", req.OrganizationFolder, req.RepositoryName, req.BranchName),
            Organization: req.OrganizationFolder,
            Repository:   req.RepositoryName,
            Branch:       req.BranchName,
            BuildNumber:  latestBuild.Number,
        })
    }
    
    return response, nil
}

6.2 DCU任务完成时记录交付物审计日志

func (s *DCUService) OnDCUTaskComplete(ctx context.Context, task *DCUTask) {
    // 记录交付物构建完成审计日志
    if s.auditWriter != nil {
        s.auditWriter.WriteJenkinsLog(&audit.JenkinsAuditLogInput{
            UserID:       task.UserID,
            Username:     task.Username,
            Action:       "DCU_COMPLETE",
            ResourceType: "dcu_artifact",
            ResourceID:   task.TaskID,
            Organization: task.Organization,
            Repository:   task.Repository,
            Branch:       task.Branch,
            BuildNumber:  task.BuildNumber,
            // 交付物信息
            ArtifactName: task.ArtifactGzipName,
            ArtifactSize: task.ArtifactSize,
            ArtifactType: "gzip",
        })
    }
}

6.3 获取AuditWriter实例

在模块初始化时获取 AuditWriter：

import (
    auditHandler "wdd.io/RMDC/rmdc-audit-log/pkg/handler"
    "wdd.io/RMDC/rmdc-audit-log/pkg/audit"
)

// 初始化时获取AuditWriter
func InitBuildService(auditDB *gorm.DB, ...) *BuildService {
    auditWriter, err := auditHandler.InitAuditWriter(auditDB)
    if err != nil {
        wdd_log.Debug("[WARN] Failed to init audit writer: %v", err)
    }
    
    return &BuildService{
        auditWriter: auditWriter,
        // ...
    }
}

7. 设计决策与优化

7.1 异步批量写入

审计日志采用缓冲区 + 批量写入策略，不阻塞业务请求：

请求处理时间 = 业务逻辑时间
审计日志写入 = 后台协程异步批量处理
  • logBuffer: 核心模块日志缓冲区 (容量: 100)
  • jenkinsLogBuffer: Jenkins模块日志缓冲区 (容量: 50)
  • userAuthLogBuffer: 用户认证模块日志缓冲区 (容量: 50)
  • 批量写入阈值: 10条
  • 定时刷新间隔: 2秒

7.2 敏感数据脱敏

sanitizeSensitiveData() 函数自动脱敏以下字段：

  • password
  • token
  • secret
  • api_key

7.3 日志清理策略

通过 /api/audit/cleanup 接口支持按保留天数清理旧日志，防止磁盘空间占用过大。

7.4 回退机制

如果 DAOManager 初始化失败，系统会自动回退到单一 DAO 模式，所有日志写入 core_audit_logs 表，保证审计功能可用性。

8. 前端审计日志查询界面

8.1 模块Tab切换

前端页面通过模块 Tab 切换查询不同表的审计日志：

flowchart LR
    A[All Modules] -->|GET /api/audit/logs| B[(core_audit_logs)]
    C[Core] -->|GET /api/audit/logs?module=core| B
    D[Jenkins] -->|GET /api/audit/logs/jenkins| E[(jenkins_audit_logs)]
    F[User Auth] -->|GET /api/audit/logs/user-auth| G[(user_auth_audit_logs)]

8.2 API 端点列表

端点 方法 描述 查询表
/api/audit/logs GET 通用审计日志查询 core_audit_logs
/api/audit/logs/jenkins GET Jenkins专属日志（含专属字段） jenkins_audit_logs
/api/audit/logs/user-auth GET 用户认证专属日志（含专属字段） user_auth_audit_logs
/api/audit/modules/stats GET 各模块日志数量统计 所有表
/api/audit/logs/stats GET 审计日志统计（大小、最旧记录） core_audit_logs
/api/audit/cleanup DELETE 清理旧日志 所有表

8.3 模块专属响应字段

Jenkins 模块

{
  "id": 1,
  "module": "jenkins",
  "username": "admin",
  "action": "TRIGGER_BUILD",
  "organization": "Backend",
  "repository": "cmii-fly-center",
  "branch": "master",
  "build_number": 123,
  "artifact_name": "cmii-fly-center-1.0.0.tar.gz",
  "artifact_size": 52428800,
  "artifact_type": "gzip"
}

UserAuth 模块

{
  "id": 1,
  "module": "user_auth",
  "username": "admin",
  "action": "LOGIN",
  "login_type": "password",
  "device_info": "Chrome/Windows"
}

8.4 前端页面功能

  1. 模块Tab切换 - 点击 Tab 动态切换查询不同模块的日志
  2. 动态表头 - 根据选中模块显示不同列（Jenkins: Organization/Repository/Branch; UserAuth: LoginType）
  3. 模块筛选 - 当选择特定模块时，显示该模块专属的筛选条件
  4. 详情弹窗 - 点击日志条目显示完整详情，包含模块专属字段

文档版本: v2.1
编制日期: 2025-12-12
编制人: System Architect
更新内容:

  • 新增 UserAuth 模块审计日志支持
  • 新增模块专属审计日志表分表写入架构
  • 新增 AuditLogDAOManager DAO管理器
  • 新增 WriteJenkinsLog/WriteUserAuthLog 接口
  • 新增模块审计日志表添加指南
  • 新增 Jenkins 交付物构建审计日志使用示例
  • 新增前端审计日志查询界面设计