# RMDC 审计日志架构设计 ## 1. 审计日志模块整体架构 ```mermaid graph TB subgraph "前端 Frontend" Vue[Vue3 前端应用] end subgraph "rmdc-core 核心模块" Router[Gin Router] CORS[CORS 中间件] Audit[审计中间件
AuditMiddleware] Auth[认证中间件
AuthMiddleware] end subgraph "业务模块层" Jenkins[rmdc-jenkins-branch-dac
Jenkins模块] UserAuth[rmdc-user-auth
用户认证模块] Watchdog[rmdc-watchdog-center
Watchdog模块] end subgraph "rmdc-audit-log 审计模块" AuditWriter[AuditWriter 接口] AuditService[AuditService
审计服务] DAOManager[AuditLogDAOManager
DAO管理器] CoreDAO[AuditLogDao
核心模块DAO] JenkinsDAO[JenkinsAuditLogDao
Jenkins模块DAO] end subgraph "数据持久层 - rmdc_audit 数据库" DB[(PostgreSQL)] CoreTable[core_audit_logs] JenkinsTable[jenkins_audit_logs] WatchdogTable[watchdog_audit_logs] UserAuthTable[user_auth_audit_logs] end Vue -->|HTTP Request| Router Router --> CORS CORS --> Audit Audit -->|记录审计日志| AuditWriter AuditWriter --> AuditService Audit --> Auth Auth --> Jenkins Auth --> UserAuth Auth --> Watchdog Jenkins -.->|WriteJenkinsLog| AuditWriter AuditService --> DAOManager DAOManager --> CoreDAO DAOManager --> JenkinsDAO CoreDAO --> DB JenkinsDAO --> DB DB --> CoreTable DB --> JenkinsTable DB --> WatchdogTable DB --> UserAuthTable style Audit fill:#ff6b6b,stroke:#c92a2a,stroke-width:2px style AuditService fill:#4ecdc4,stroke:#087f5b,stroke-width:2px style DAOManager fill:#74c0fc,stroke:#1864ab,stroke-width:2px style DB fill:#ffd43b,stroke:#f08c00,stroke-width:2px ``` --- ## 2. Jenkins 模块审计日志写入流程 以触发 Jenkins 构建为例，展示一条请求如何被正确写入 `jenkins_audit_logs` 表： ```mermaid sequenceDiagram participant U as 用户浏览器 participant F as Vue3 前端 participant R as Gin Router participant AM as AuditMiddleware participant Auth as AuthMiddleware participant JH as Jenkins Handler participant JS as Jenkins Service participant AS as AuditService participant JenkinsDAO as JenkinsAuditLogDao participant DB as PostgreSQL rect rgb(220, 240, 255) Note over U,DB: 阶段1: 请求发起与审计捕获 end U->>F: 点击"触发构建"按钮 F->>R: POST /api/builds/trigger
{organization, repository, branch} R->>AM: 请求进入中间件链 activate AM AM->>AM: 记录开始时间 startTime AM->>AM: 读取并缓存 RequestBody AM->>AM: 提取 QueryParams rect rgb(255, 240, 220) Note over AM,Auth: 阶段2: 认证与业务处理 end AM->>Auth: c.Next() 继续处理 Auth->>Auth: 验证JWT Token Auth->>Auth: 提取 user_id, username Auth->>JH: 路由到 Jenkins Handler JH->>JS: TriggerBuild(org, repo, branch) JS->>JS: 调用 Jenkins API JS-->>JH: 返回构建结果 JH-->>Auth: 响应 200 OK Auth-->>AM: 返回控制权 rect rgb(220, 255, 220) Note over AM,DB: 阶段3: 审计日志异步写入 end AM->>AM: 计算 duration = time.Since(startTime) AM->>AM: 从 context 提取 user_id, username AM->>AM: 提取 resourceType=jenkins_project AM->>AM: determineAction() → "TRIGGER_BUILD" AM->>AM: sanitizeSensitiveData(requestBody) AM->>AS: WriteLog(auditLog) 写入Jenkins缓冲区 deactivate AM activate AS AS->>AS: 判断 Module="jenkins" AS->>AS: 放入 jenkinsLogBuffer AS->>AS: jenkinsBatchWriter 批量处理 AS->>JenkinsDAO: BatchCreate(ctx, logs) JenkinsDAO->>DB: INSERT INTO jenkins_audit_logs DB-->>JenkinsDAO: 写入成功 deactivate AS Note over DB: jenkins_audit_logs 表中新增一条记录:
action=TRIGGER_BUILD
organization=Backend
repository=cmii-fly-center
branch=feature-xxx ``` ### 关键代码路径 | 步骤 | 文件 | 函数/方法 | 说明 | |------|------|-----------|------| | 1 | `pkg/middleware/audit_middleware.go` | `AuditMiddleware()` | 中间件入口，捕获请求信息 | | 2 | `pkg/middleware/audit_middleware.go` | `determineAction()` | 根据 HTTP 方法和路径确定操作类型 | | 3 | `pkg/middleware/audit_middleware.go` | `extractResourceInfo()` | 提取资源类型和资源ID | | 4 | `pkg/middleware/audit_middleware.go` | `sanitizeSensitiveData()` | 脱敏处理敏感数据 | | 5 | `internal/service/audit_service.go` | `WriteLog()` | 实现 AuditWriter 接口，按模块路由到不同缓冲区 | | 6 | `internal/service/audit_service.go` | `jenkinsBatchWriter()` | Jenkins专用批量写入协程 | | 7 | `internal/dao/jenkins_audit_log_dao.go` | `BatchCreate()` | Jenkins审计日志批量插入 | --- ## 3. 多模块审计日志记录方式 ### 3.1 审计日志表结构 RMDC 采用**分表存储**策略，每个业务模块有独立的审计日志表，继承公共基础字段并扩展模块特定字段： ```mermaid erDiagram AuditLog { int64 id PK string module int64 user_id FK string username string action string resource_type string resource_id text details string ip_address string status text error_message datetime created_at } JenkinsAuditLog { int64 id PK string organization string repository string branch int build_number string artifact_name int64 artifact_size string artifact_type } WatchdogAuditLog { int64 id PK string project_id string project_name text host_info } UserAuthAuditLog { int64 id PK string login_type string device_info } AuditLog ||--|| JenkinsAuditLog : "继承" AuditLog ||--|| WatchdogAuditLog : "继承" AuditLog ||--|| UserAuthAuditLog : "继承" ``` ### 3.2 模块与表的对应关系 | 模块 | 表名 | 扩展字段 | 典型操作 | |------|------|----------|----------| | Core | `core_audit_logs` | 无 | 系统配置修改、审计日志清理 | | Jenkins | `jenkins_audit_logs` | organization, repository, branch, build_number, artifact_name, artifact_size, artifact_type | 触发构建、同步Jenkins、DCU任务、交付物构建 | | Watchdog | `watchdog_audit_logs` | project_id, project_name, host_info | 主机授权、项目注册 | | UserAuth | `user_auth_audit_logs` | login_type, device_info | 用户创建、角色修改 | ### 3.3 DAO 管理器架构 ```mermaid classDiagram class AuditLogDAOManager { +Core *AuditLogDao +Jenkins *JenkinsAuditLogDao +GetDAO(module string) interface{} } class AuditLogDao { -db *gorm.DB +Create(ctx, log) error +BatchCreate(ctx, logs) error +List(ctx, module, page, pageSize) ([]*AuditLog, int64, error) } class JenkinsAuditLogDao { -db *gorm.DB +Create(ctx, log) error +BatchCreate(ctx, logs) error +List(ctx, page, pageSize) ([]*JenkinsAuditLog, int64, error) +ListByOrganization(ctx, org, page, pageSize) ([]*JenkinsAuditLog, int64, error) } AuditLogDAOManager --> AuditLogDao AuditLogDAOManager --> JenkinsAuditLogDao ``` ### 3.4 Jenkins 审计日志写入流程 ```mermaid flowchart TD A[HTTP 请求到达] --> B{路径包含 /jenkins/ 或 /builds/ 或 /dcu/?} B -->|是| C[AuditMiddleware 捕获请求] B -->|否| D[其他模块处理] C --> E[提取 JWT 中的 user_id, username] E --> F[解析路径参数: org, repo, branch] F --> G[determineAction: TRIGGER_BUILD / SYNC_JENKINS / DCU_START] G --> H[构建 audit.AuditLog 对象] H --> I[调用 AuditWriter.WriteLog] I --> J[AuditService.WriteLog] J --> K{判断模块类型 + DAOManager存在?} K -->|jenkins + 有DAOManager| L[构建 JenkinsAuditLog] K -->|其他| M[构建 AuditLog] L --> N[放入 jenkinsLogBuffer] N --> O[jenkinsBatchWriter 批量处理] O --> P[JenkinsAuditLogDao.BatchCreate] P --> Q[(写入 jenkins_audit_logs 表)] M --> R[放入 logBuffer] R --> S[batchWriter 批量处理] S --> T[AuditLogDao.BatchCreate] T --> U[(写入 core_audit_logs 表)] style C fill:#ff6b6b,stroke:#c92a2a style I fill:#4ecdc4,stroke:#087f5b style Q fill:#ffd43b,stroke:#f08c00 style U fill:#ffd43b,stroke:#f08c00 ``` --- ## 4. AuditWriter 接口实现 ### 4.1 接口定义 位置: `rmdc-audit-log/pkg/audit/interface.go` ```go // AuditLog 审计日志结构（公共接口） type AuditLog struct { UserID int64 `json:"user_id"` Username string `json:"username"` Action string `json:"action"` ResourceType string `json:"resource_type"` ResourceID string `json:"resource_id"` IPAddress string `json:"ip_address"` Method string `json:"method"` Path string `json:"path"` QueryParams string `json:"query_params"` RequestBody string `json:"request_body"` Duration int64 `json:"duration"` Module string `json:"module,omitempty"` // 显式指定模块（可选） } // AuditWriter 审计日志写入接口 type AuditWriter interface { // WriteLog 写入审计日志（通用方法，根据Path自动判断模块） WriteLog(log *AuditLog) // WriteJenkinsLog 写入Jenkins模块审计日志 // 用于需要记录Jenkins专属字段的场景 WriteJenkinsLog(log *JenkinsAuditLogInput) } // JenkinsAuditLogInput Jenkins审计日志输入结构 type JenkinsAuditLogInput struct { // 通用字段 UserID int64 `json:"user_id"` Username string `json:"username"` Action string `json:"action"` ResourceType string `json:"resource_type"` ResourceID string `json:"resource_id"` IPAddress string `json:"ip_address"` Details string `json:"details"` // Jenkins专属字段 Organization string `json:"organization"` Repository string `json:"repository"` Branch string `json:"branch"` BuildNumber int `json:"build_number"` // 交付物相关字段 ArtifactName string `json:"artifact_name"` ArtifactSize int64 `json:"artifact_size"` ArtifactType string `json:"artifact_type"` // docker_image, gzip, jar等 } ``` ### 4.2 模块判断逻辑 ```go func (s *AuditService) determineModule(path string) string { switch { case strings.Contains(path, "/jenkins/") || strings.Contains(path, "/builds/") || strings.Contains(path, "/projects/") || strings.Contains(path, "/dcu/"): return "jenkins" case strings.Contains(path, "/users/") || strings.Contains(path, "/auth/") || strings.Contains(path, "/permissions/"): return "user_auth" case strings.Contains(path, "/watchdog/"): return "watchdog" default: return "core" } } ``` --- ## 5. 新模块审计日志表添加指南 如需为新模块添加审计日志表，请按以下步骤操作： ### 5.1 步骤概览 ```mermaid flowchart LR A[1. 定义Entity] --> B[2. 创建DAO] B --> C[3. 更新DAOManager] C --> D[4. 扩展Service] D --> E[5. 更新模块判断] E --> F[6. 数据库迁移] ``` ### 5.2 详细步骤 #### 步骤1: 定义实体 (Entity) 位置: `rmdc-audit-log/internal/model/entity/audit_log.go` ```go // NewModuleAuditLog 新模块审计日志 type NewModuleAuditLog struct { AuditLog // 嵌入基础审计日志 // 添加模块专属字段 CustomField1 string `gorm:"size:255" json:"custom_field_1"` CustomField2 int64 `json:"custom_field_2"` } // TableName 指定表名 func (NewModuleAuditLog) TableName() string { return "new_module_audit_logs" } ``` #### 步骤2: 创建DAO 位置: `rmdc-audit-log/internal/dao/new_module_audit_log_dao.go` ```go type NewModuleAuditLogDao struct { db *gorm.DB } func NewNewModuleAuditLogDao(db *gorm.DB) *NewModuleAuditLogDao { return &NewModuleAuditLogDao{db: db} } func (d *NewModuleAuditLogDao) Create(ctx context.Context, log *entity.NewModuleAuditLog) error { log.Module = "new_module" return d.db.WithContext(ctx).Create(log).Error } func (d *NewModuleAuditLogDao) BatchCreate(ctx context.Context, logs []*entity.NewModuleAuditLog) error { if len(logs) == 0 { return nil } for i := range logs { logs[i].Module = "new_module" } return d.db.WithContext(ctx).CreateInBatches(logs, len(logs)).Error } ``` #### 步骤3: 更新DAOManager 位置: `rmdc-audit-log/internal/dao/audit_log_dao_manager.go` ```go type AuditLogDAOManager struct { Core *AuditLogDao Jenkins *JenkinsAuditLogDao NewModule *NewModuleAuditLogDao // 新增 } func InitAuditLogDAOManager(db *gorm.DB) (*AuditLogDAOManager, error) { // ... daoManagerInstance = &AuditLogDAOManager{ Core: NewAuditLogDao(db), Jenkins: NewJenkinsAuditLogDao(db), NewModule: NewNewModuleAuditLogDao(db), // 新增 } // ... } ``` #### 步骤4: 扩展Service 位置: `rmdc-audit-log/internal/service/audit_service.go` 1. 添加新模块的日志缓冲区 2. 添加新模块的批量写入协程 3. 在 `WriteLog` 中添加新模块的路由逻辑 #### 步骤5: 更新模块判断逻辑 ```go func (s *AuditService) determineModule(path string) string { switch { // ... 现有逻辑 ... case strings.Contains(path, "/new-module/"): return "new_module" default: return "core" } } ``` #### 步骤6: 数据库迁移 位置: `rmdc-core/scripts/init_postgresql.sql` ```sql -- 新模块审计日志表 CREATE TABLE IF NOT EXISTS new_module_audit_logs ( id BIGSERIAL PRIMARY KEY, module VARCHAR(50) DEFAULT 'new_module', user_id INTEGER, username VARCHAR(64), action VARCHAR(64) NOT NULL, resource_type VARCHAR(50), resource_id VARCHAR(255), details JSONB, ip_address INET, status VARCHAR(20), error_message TEXT, created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP, -- 模块专属字段 custom_field_1 VARCHAR(255), custom_field_2 BIGINT ); CREATE INDEX idx_new_module_audit_created ON new_module_audit_logs (created_at DESC); ``` --- ## 6. Jenkins 交付物构建审计日志使用示例 ### 6.1 触发构建时记录审计日志 位置: `rmdc-jenkins-branch-dac/internal/service/build_service.go` ```go func (s *BuildService) TriggerBuild(ctx context.Context, req *dto.TriggerBuildRequest) (*dto.TriggerBuildResponse, error) { // ... 构建触发逻辑 ... // 记录Jenkins审计日志 if s.auditWriter != nil { s.auditWriter.WriteJenkinsLog(&audit.JenkinsAuditLogInput{ UserID: getUserIDFromContext(ctx), Username: getUsernameFromContext(ctx), Action: "TRIGGER_BUILD", ResourceType: "jenkins_build", ResourceID: fmt.Sprintf("%s/%s/%s", req.OrganizationFolder, req.RepositoryName, req.BranchName), Organization: req.OrganizationFolder, Repository: req.RepositoryName, Branch: req.BranchName, BuildNumber: latestBuild.Number, }) } return response, nil } ``` ### 6.2 DCU任务完成时记录交付物审计日志 ```go func (s *DCUService) OnDCUTaskComplete(ctx context.Context, task *DCUTask) { // 记录交付物构建完成审计日志 if s.auditWriter != nil { s.auditWriter.WriteJenkinsLog(&audit.JenkinsAuditLogInput{ UserID: task.UserID, Username: task.Username, Action: "DCU_COMPLETE", ResourceType: "dcu_artifact", ResourceID: task.TaskID, Organization: task.Organization, Repository: task.Repository, Branch: task.Branch, BuildNumber: task.BuildNumber, // 交付物信息 ArtifactName: task.ArtifactGzipName, ArtifactSize: task.ArtifactSize, ArtifactType: "gzip", }) } } ``` ### 6.3 获取AuditWriter实例 在模块初始化时获取 AuditWriter： ```go import ( auditHandler "wdd.io/RMDC/rmdc-audit-log/pkg/handler" "wdd.io/RMDC/rmdc-audit-log/pkg/audit" ) // 初始化时获取AuditWriter func InitBuildService(auditDB *gorm.DB, ...) *BuildService { auditWriter, err := auditHandler.InitAuditWriter(auditDB) if err != nil { wdd_log.Debug("[WARN] Failed to init audit writer: %v", err) } return &BuildService{ auditWriter: auditWriter, // ... } } ``` --- ## 7. 设计决策与优化 ### 7.1 异步批量写入 审计日志采用**缓冲区 + 批量写入**策略，不阻塞业务请求： ``` 请求处理时间 = 业务逻辑时间 审计日志写入 = 后台协程异步批量处理 ``` - `logBuffer`: 核心模块日志缓冲区 (容量: 100) - `jenkinsLogBuffer`: Jenkins模块日志缓冲区 (容量: 50) - `userAuthLogBuffer`: 用户认证模块日志缓冲区 (容量: 50) - 批量写入阈值: 10条 - 定时刷新间隔: 2秒 ### 7.2 敏感数据脱敏 `sanitizeSensitiveData()` 函数自动脱敏以下字段： - `password` - `token` - `secret` - `api_key` ### 7.3 日志清理策略 通过 `/api/audit/cleanup` 接口支持按保留天数清理旧日志，防止磁盘空间占用过大。 ### 7.4 回退机制 如果 DAOManager 初始化失败，系统会自动回退到单一 DAO 模式，所有日志写入 `core_audit_logs` 表，保证审计功能可用性。 --- ## 8. 前端审计日志查询界面 ### 8.1 模块Tab切换 前端页面通过模块 Tab 切换查询不同表的审计日志： ```mermaid flowchart LR A[All Modules] -->|GET /api/audit/logs| B[(core_audit_logs)] C[Core] -->|GET /api/audit/logs?module=core| B D[Jenkins] -->|GET /api/audit/logs/jenkins| E[(jenkins_audit_logs)] F[User Auth] -->|GET /api/audit/logs/user-auth| G[(user_auth_audit_logs)] ``` ### 8.2 API 端点列表 | 端点 | 方法 | 描述 | 查询表 | |------|------|------|--------| | `/api/audit/logs` | GET | 通用审计日志查询 | core_audit_logs | | `/api/audit/logs/jenkins` | GET | Jenkins专属日志（含专属字段） | jenkins_audit_logs | | `/api/audit/logs/user-auth` | GET | 用户认证专属日志（含专属字段） | user_auth_audit_logs | | `/api/audit/modules/stats` | GET | 各模块日志数量统计 | 所有表 | | `/api/audit/logs/stats` | GET | 审计日志统计（大小、最旧记录） | core_audit_logs | | `/api/audit/cleanup` | DELETE | 清理旧日志 | 所有表 | ### 8.3 模块专属响应字段 #### Jenkins 模块 ```json { "id": 1, "module": "jenkins", "username": "admin", "action": "TRIGGER_BUILD", "organization": "Backend", "repository": "cmii-fly-center", "branch": "master", "build_number": 123, "artifact_name": "cmii-fly-center-1.0.0.tar.gz", "artifact_size": 52428800, "artifact_type": "gzip" } ``` #### UserAuth 模块 ```json { "id": 1, "module": "user_auth", "username": "admin", "action": "LOGIN", "login_type": "password", "device_info": "Chrome/Windows" } ``` ### 8.4 前端页面功能 1. **模块Tab切换** - 点击 Tab 动态切换查询不同模块的日志 2. **动态表头** - 根据选中模块显示不同列（Jenkins: Organization/Repository/Branch; UserAuth: LoginType） 3. **模块筛选** - 当选择特定模块时，显示该模块专属的筛选条件 4. **详情弹窗** - 点击日志条目显示完整详情，包含模块专属字段 --- **文档版本**: v2.1 **编制日期**: 2025-12-12 **编制人**: System Architect **更新内容**: - 新增 UserAuth 模块审计日志支持 - 新增模块专属审计日志表分表写入架构 - 新增 AuditLogDAOManager DAO管理器 - 新增 WriteJenkinsLog/WriteUserAuthLog 接口 - 新增模块审计日志表添加指南 - 新增 Jenkins 交付物构建审计日志使用示例 - 新增前端审计日志查询界面设计