使用Gitleaks检测代码库中的敏感信息

技术背景

在软件开发过程中，代码库中可能会意外包含敏感信息，如密码、API密钥和令牌等。这些信息的泄露可能会导致严重的安全问题。Gitleaks是一个用于检测Git仓库、文件以及通过标准输入传递的任何内容中敏感信息的工具。

实现步骤

安装

Gitleaks可以通过多种方式安装：

Homebrew（MacOS）：

1	`brew install gitleaks`

Docker（DockerHub）：

1 2	`docker pull zricethezav/gitleaks:latest docker run -v ${path_to_host_folder_to_scan}:/path zricethezav/gitleaks:latest [COMMAND] [OPTIONS] [SOURCE_PATH]`

Docker（ghcr.io）：

1 2	`docker pull ghcr.io/gitleaks/gitleaks:latest docker run -v ${path_to_host_folder_to_scan}:/path ghcr.io/gitleaks/gitleaks:latest [COMMAND] [OPTIONS] [SOURCE_PATH]`

从源代码安装（需安装Go）：

1
2
3

git clone https://github.com/gitleaks/gitleaks.git
cd gitleaks
make build

作为GitHub Action使用

name: gitleaks
on: [pull_request, push, workflow_dispatch]
jobs:
  scan:
    name: gitleaks
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE}} # 仅组织账户需要，个人账户不需要

作为pre-commit钩子使用

从https://pre-commit.com/#install安装pre-commit。
在仓库根目录创建.pre-commit-config.yaml文件，内容如下：

repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.24.2
    hooks:
      - id: gitleaks

执行pre-commit autoupdate自动更新配置到最新版本。
执行pre-commit install进行安装。

核心代码

配置文件示例

title = "Custom Gitleaks configuration"

[extend]
useDefault = true
disabledRules = [ "generic-api-key"]

[[rules]]
id = "awesome-rule-1"
description = "awesome rule 1"
regex = '''one-go-style-regex-for-this-rule'''
secretGroup = 3
entropy = 3.5
path = '''a-file-path-regex'''
keywords = [
  "auth",
  "password",
  "token",
]
tags = ["tag","another tag"]

[[rules.allowlists]]
description = "ignore commit A"
condition = "OR"
commits = [ "commit-A", "commit-B"]
paths = [
  '''go\.mod''',
  '''go\.sum'''
]
stopwords = [
  '''client''',
  '''endpoint''',
]

[[rules.allowlists]]
condition = "AND"
regexTarget = "match"
regexes = [ '''(?i)parseur[il]''' ]
paths = [ '''package-lock\.json''' ]

[[allowlists]]
description = "global allow list"
commits = [ "commit-A", "commit-B", "commit-C"]
paths = [
  '''gitleaks\.toml''',
  '''(.*?)(jpg|gif|doc)'''
]
regexTarget = "match"
regexes = [
  '''219-09-9999''',
  '''078-05-1120''',
  '''(9[0-9]{2}|666)-\d{2}-\d{4}''',
]
stopwords = [
  '''client''',
  '''endpoint''',
]

[[allowlists]]
targetRules = ["awesome-rule-1", "awesome-rule-2"]
description = "Our test assets trigger false-positives in a couple rules."
paths = ['''tests/expected/._\.json$''']

自定义报告模板示例（jsonextra.tmpl）

[{{ $lastFinding := (sub (len . ) 1) }}
{{- range $i, $finding := . }}{{with $finding}}
    {
        "Description": {{ quote .Description }},
        "StartLine": {{ .StartLine }},
        "EndLine": {{ .EndLine }},
        "StartColumn": {{ .StartColumn }},
        "EndColumn": {{ .EndColumn }},
        "Line": {{ quote .Line }},
        "Match": {{ quote .Match }},
        "Secret": {{ quote .Secret }},
        "File": "{{ .File }}",
        "SymlinkFile": {{ quote .SymlinkFile }},
        "Commit": {{ quote .Commit }},
        "Entropy": {{ .Entropy }},
        "Author": {{ quote .Author }},
        "Email": {{ quote .Email }},
        "Date": {{ quote .Date }},
        "Message": {{ quote .Message }},
        "Tags": [{{ $lastTag := (sub (len .Tags ) 1) }}{{ range $j, $tag := .Tags }}{{ quote . }}{{ if ne $j $lastTag }},{{ end }}{{ end }}],
        "RuleID": {{ quote .RuleID }},
        "Fingerprint": {{ quote .Fingerprint }}
    }{{ if ne $i $lastFinding }},{{ end }}
{{- end}}{{ end }}
]

最佳实践

创建基线：对于大型仓库或历史较长的仓库，使用基线可以忽略旧的检测结果。通过--report-path参数生成报告作为基线，再次检测时使用--baseline-path应用基线。

1 2	`gitleaks git --report-path gitleaks-report.json gitleaks git --baseline-path gitleaks-report.json --report-path findings.json`

自定义规则：根据项目需求自定义检测规则，可参考默认配置文件https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml。
使用报告模板：如果内置报告格式不满足需求，可使用--report-template创建自定义报告。