openclaw-memory/DEPLOY.md

# 🚀 龙虾记忆同步系统 - 部署文档

完整的部署指南，涵盖从开发环境到生产环境的所有步骤。

## 📋 目录

- [系统要求](#系统要求)
- [环境准备](#环境准备)
- [快速部署](#快速部署)
- [生产环境部署](#生产环境部署)
- [数据库管理](#数据库管理)
- [监控与维护](#监控与维护)
- [故障排查](#故障排查)
- [备份与恢复](#备份与恢复)
- [性能优化](#性能优化)
- [安全加固](#安全加固)

---

## 💻 系统要求

### 最低配置

| 项目 | 要求 |
|------|------|
| 操作系统 | Linux / macOS / Windows (WSL2) |
| CPU | 2 核 |
| 内存 | 2GB RAM |
| 磁盘 | 5GB 可用空间 |
| 网络 | 稳定的网络连接 |

### 推荐配置

| 项目 | 要求 |
|------|------|
| 操作系统 | Ubuntu 22.04 LTS / CentOS 8+ |
| CPU | 4 核 |
| 内存 | 4GB RAM |
| 磁盘 | 20GB SSD |
| 网络 | 千兆网络 |

### 端口要求

| 端口 | 服务 | 说明 |
|------|------|------|
| 8086 | 前端 | React 应用 |
| 8087 | 后端 API | Django REST API |
| 5432 | PostgreSQL | 数据库 |
| 443 | HTTPS | Nginx（生产环境） |

---

## 🔧 环境准备

### 1. 安装 Docker

#### Ubuntu / Debian

```bash
# 更新包索引
sudo apt-get update

# 安装依赖
sudo apt-get install -y ca-certificates curl gnupg lsb-release

# 添加 Docker 官方 GPG key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

# 添加 Docker 仓库
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# 安装 Docker
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

# 验证安装
docker --version
docker compose version
```

#### CentOS / RHEL

```bash
# 安装依赖
sudo yum install -y yum-utils device-mapper-persistent-data lvm2

# 添加 Docker 仓库
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

# 安装 Docker
sudo yum install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

# 启动 Docker
sudo systemctl start docker
sudo systemctl enable docker
```

#### macOS

```bash
# 使用 Homebrew 安装
brew install --cask docker

# 启动 Docker Desktop
open -a Docker
```

### 2. 配置 Docker 用户组（可选但推荐）

```bash
# 将当前用户添加到 docker 组
sudo usermod -aG docker $USER

# 重新登录或运行
newgrp docker

# 验证
docker ps
```

### 3. 安装 Git

```bash
# Ubuntu / Debian
sudo apt-get install -y git

# CentOS / RHEL
sudo yum install -y git

# macOS
brew install git
```

### 4. 安装 Nginx（生产环境）

```bash
# Ubuntu / Debian
sudo apt-get install -y nginx

# CentOS / RHEL
sudo yum install -y nginx
```

---

## 🚀 快速部署

### 1. 克隆项目

```bash
# 克隆仓库
git clone http://10.2.0.100:8989/daotong/lobster-memory-sync.git
cd lobster-memory-sync
```

### 2. 配置环境变量

创建 `.env` 文件：

```bash
# 数据库配置
DB_NAME=lobster_memory
DB_USER=postgres
DB_PASSWORD=your_secure_password_here
DB_HOST=postgres
DB_PORT=5432

# 龙虾记忆目录路径
LOBSTER_MEMORY_BASE=/home/node/.openclaw/workspace/daotong

# 前端配置
REACT_APP_API_URL=http://localhost:8087/api

# 端口配置
FRONTEND_PORT=8086
BACKEND_PORT=8087
POSTGRES_PORT=5432

# Docker 配置
COMPOSE_PROJECT_NAME=lobster-memory-sync
```

**重要提示：**
- 修改 `DB_PASSWORD` 为强密码
- 修改 `LOBSTER_MEMORY_BASE` 为实际的龙虾记忆目录路径
- 生产环境建议使用 `REACT_APP_API_URL=https://yourdomain.com/api`

### 3. 修改 docker-compose.yml

编辑 `docker-compose.yml`：

```yaml
version: '3.8'

services:
  postgres:
    image: postgres:15-alpine
    container_name: lobster-postgres
    environment:
      POSTGRES_DB: ${DB_NAME}
      POSTGRES_USER: ${DB_USER}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "${POSTGRES_PORT:-5432}:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"]
      interval: 10s
      timeout: 5s
      retries: 5

  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    container_name: lobster-backend
    environment:
      DB_HOST: ${DB_HOST}
      DB_NAME: ${DB_NAME}
      DB_USER: ${DB_USER}
      DB_PASSWORD: ${DB_PASSWORD}
      DB_PORT: ${DB_PORT}
      LOBSTER_MEMORY_BASE: ${LOBSTER_MEMORY_BASE}
      DJANGO_SETTINGS_MODULE: memory_sync.settings
    volumes:
      # 挂载龙虾记忆目录（只读）
      - ${LOBSTER_MEMORY_BASE}:/app/memory_files:ro
      # 代码热重载（开发用）
      - ./backend:/app
    ports:
      - "${BACKEND_PORT:-8087}:8087"
    depends_on:
      postgres:
        condition: service_healthy
    command: >
      sh -c "
        python manage.py migrate &&
        python manage.py runserver 0.0.0.0:8087
      "
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8087/api/"]
      interval: 30s
      timeout: 10s
      retries: 3

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    container_name: lobster-frontend
    ports:
      - "${FRONTEND_PORT:-8086}:80"
    environment:
      - REACT_APP_API_URL=${REACT_APP_API_URL}
    depends_on:
      - backend

volumes:
  postgres_data:
```

### 4. 构建并启动服务

```bash
# 构建镜像
docker-compose build

# 启动所有服务（后台运行）
docker-compose up -d

# 查看服务状态
docker-compose ps

# 查看日志
docker-compose logs -f
```

**预期输出：**
```
NAME                    STATUS
lobster-postgres        Up (healthy)
lobster-backend         Up (healthy)
lobster-frontend        Up
```

### 5. 初始化数据库

```bash
# 等待数据库启动
sleep 10

# 执行数据库迁移
docker-compose exec backend python manage.py migrate

# 创建超级用户（可选）
docker-compose exec backend python manage.py createsuperuser

# 验证迁移
docker-compose exec backend python manage.py showmigrations memory_app
```

**预期输出：**
```
[X] 0001_initial
[X] 0002_add_summary_and_audit_fields
[X] 0003_add_file_attribute
```

### 6. 验证部署

#### 检查服务状态

```bash
# 查看所有容器状态
docker-compose ps

# 查看容器资源使用
docker stats
```

#### 测试后端 API

```bash
# 测试 API 健康检查
curl http://localhost:8087/api/

# 测试文件扫描
curl "http://localhost:8087/api/scan/?lobster_id=daotong"

# 测试同步状态
curl "http://localhost:8087/api/status/?lobster_id=daotong"
```

#### 访问前端

打开浏览器访问：
- http://localhost:8086

**预期效果：**
- ✅ 能够看到文件树展示
- ✅ 能够点击文件查看差异对比
- ✅ 能够执行同步操作
- ✅ 能够查看操作历史

---

## 🏭 生产环境部署

### 1. 使用环境变量文件

创建 `.env.production` 文件：

```bash
# 生产环境配置
DB_NAME=lobster_memory_prod
DB_USER=postgres
DB_PASSWORD=<强密码，至少 32 位>
DB_HOST=postgres
DB_PORT=5432

# 龙虾记忆目录
LOBSTER_MEMORY_BASE=/var/lib/lobster/memory

# 前端 API 地址
REACT_APP_API_URL=https://api.yourdomain.com/api

# 端口配置
FRONTEND_PORT=8086
BACKEND_PORT=8087
POSTGRES_PORT=5432

# Docker 配置
COMPOSE_PROJECT_NAME=lobster-memory-sync-prod
```

### 2. 配置 Nginx 反向代理

创建 Nginx 配置文件 `/etc/nginx/sites-available/lobster-memory-sync`：

```nginx
# 上游服务器配置
upstream lobster_backend {
    server localhost:8087;
    keepalive 32;
}

upstream lobster_frontend {
    server localhost:8086;
    keepalive 32;
}

# HTTP 重定向到 HTTPS
server {
    listen 80;
    listen [::]:80;
    server_name yourdomain.com;

    # Let's Encrypt 验证
    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    # 重定向到 HTTPS
    location / {
        return 301 https://$server_name$request_uri;
    }
}

# HTTPS 配置
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name yourdomain.com;

    # SSL 证书配置
    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384';
    ssl_prefer_server_ciphers off;

    # HSTS
    add_header Strict-Transport-Security "max-age=63072000" always;

    # 安全头部
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    # 日志
    access_log /var/log/nginx/lobster-memory-access.log;
    error_log /var/log/nginx/lobster-memory-error.log;

    # 前端静态资源
    location / {
        proxy_pass http://lobster_frontend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # 缓存配置
        expires 1h;
        add_header Cache-Control "public, immutable";
    }

    # 后端 API
    location /api/ {
        proxy_pass http://lobster_backend/api/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # 超时配置
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;

        # 禁用缓存
        add_header Cache-Control "no-cache, no-store, must-revalidate";
    }

    # WebSocket 支持（如果需要）
    location /ws/ {
        proxy_pass http://lobster_backend/ws/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}
```

启用配置：

```bash
# 创建符号链接
sudo ln -s /etc/nginx/sites-available/lobster-memory-sync /etc/nginx/sites-enabled/

# 测试配置
sudo nginx -t

# 重启 Nginx
sudo systemctl restart nginx
```

### 3. 启用 HTTPS

使用 Let's Encrypt 获取免费 SSL 证书：

```bash
# 安装 certbot
sudo apt-get install certbot python3-certbot-nginx

# 获取证书
sudo certbot --nginx -d yourdomain.com

# 测试自动续期
sudo certbot renew --dry-run

# 添加自动续期任务
sudo crontab -e
```

添加以下行：

```
0 0 * * * certbot renew --quiet --post-hook "systemctl reload nginx"
```

### 4. 配置防火墙

```bash
# Ubuntu (UFW)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

# CentOS (firewalld)
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
```

### 5. 配置数据库备份

创建备份脚本 `/usr/local/bin/lobster-backup.sh`：

```bash
#!/bin/bash

# 配置
BACKUP_DIR="/var/backups/lobster-memory"
COMPOSE_FILE="/opt/lobster-memory-sync/docker-compose.yml"
DB_NAME="lobster_memory_prod"
DB_USER="postgres"
RETENTION_DAYS=30

# 创建备份目录
mkdir -p $BACKUP_DIR

# 生成备份文件名
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="$BACKUP_DIR/lobster_memory_$DATE.sql.gz"

# 执行备份
docker-compose -f $COMPOSE_FILE exec -T postgres pg_dump -U $DB_USER $DB_NAME | gzip > $BACKUP_FILE

# 检查备份是否成功
if [ $? -eq 0 ]; then
    echo "Backup completed: $BACKUP_FILE"

    # 删除旧备份
    find $BACKUP_DIR -name "lobster_memory_*.sql.gz" -mtime +$RETENTION_DAYS -delete

    # 记录日志
    logger "Lobster Memory backup completed: $BACKUP_FILE"
else
    echo "Backup failed!"
    logger "Lobster Memory backup failed!"
    exit 1
fi
```

添加执行权限：

```bash
sudo chmod +x /usr/local/bin/lobster-backup.sh
```

添加定时任务：

```bash
sudo crontab -e
```

添加以下行：

```
# 每天凌晨 2 点执行备份
0 2 * * * /usr/local/bin/lobster-backup.sh >> /var/log/lobster-backup.log 2>&1
```

### 6. 配置日志轮转

创建日志轮转配置 `/etc/logrotate.d/lobster-memory`：

```
/var/log/nginx/lobster-memory-*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 644 www-data www-data
    sharedscripts
    postrotate
        systemctl reload nginx > /dev/null 2>&1 || true
    endscript
}

/var/log/lobster-backup.log {
    weekly
    rotate 12
    compress
    missingok
    notifempty
    create 644 root root
}
```

---

## 🗄️ 数据库管理

### 连接数据库

```bash
# 进入数据库容器
docker-compose exec postgres psql -U postgres -d lobster_memory

# 或使用主机连接
psql -h localhost -U postgres -d lobster_memory
```

### 常用 SQL 命令

```sql
-- 查看所有表
\dt

-- 查看表结构
\d lobster_memory

-- 查看索引
\d lobster_memory

-- 查看同步历史
SELECT * FROM sync_history ORDER BY created_at DESC LIMIT 10;

-- 查看文件属性
SELECT * FROM file_attribute WHERE lobster_id = 'daotong';

-- 统计文件数量
SELECT COUNT(*) FROM lobster_memory WHERE lobster_id = 'daotong';

-- 查看冲突文件
SELECT file_path, status, version FROM lobster_memory WHERE status IN ('conflict', 'hard_conflict');
```

### 数据库性能优化

```sql
-- 创建索引
CREATE INDEX idx_lobster_memory_lobster_updated ON lobster_memory(lobster_id, updated_at);
CREATE INDEX idx_sync_history_lobster_updated ON sync_history(lobster_id, created_at);

-- 分析表
ANALYZE lobster_memory;
ANALYZE sync_history;
ANALYZE file_attribute;

-- 清理死元组
VACUUM FULL lobster_memory;
```

---

## 📊 监控与维护

### 查看服务日志

```bash
# 查看所有服务日志
docker-compose logs -f

# 查看特定服务日志
docker-compose logs -f backend
docker-compose logs -f frontend
docker-compose logs -f postgres

# 查看最近 100 行日志
docker-compose logs --tail=100 backend

# 查看特定时间范围的日志
docker-compose logs --since=2024-01-01T00:00:00 backend
```

### 查看资源使用

```bash
# 查看容器资源使用情况
docker stats

# 查看磁盘使用
docker system df

# 清理未使用的资源
docker system prune -a

# 查看容器详细信息
docker inspect lobster-backend
```

### 健康检查

```bash
# 检查后端 API
curl -f http://localhost:8087/api/ || echo "Backend is down"

# 检查前端
curl -f http://localhost:8086/ || echo "Frontend is down"

# 检查数据库
docker-compose exec postgres pg_isready -U postgres
```

### 自动化监控脚本

创建监控脚本 `/usr/local/bin/lobster-monitor.sh`：

```bash
#!/bin/bash

# 配置
WEBHOOK_URL="https://your-webhook-url.com"
COMPOSE_FILE="/opt/lobster-memory-sync/docker-compose.yml"

# 检查服务状态
check_service() {
    local service=$1
    local url=$2

    if ! curl -f -s -o /dev/null "$url"; then
        echo "Service $service is down!"

        # 发送告警
        curl -X POST "$WEBHOOK_URL" \
            -H "Content-Type: application/json" \
            -d "{\"text\": \"🚨 Service $service is down!\"}"

        # 重启服务
        docker-compose -f $COMPOSE_FILE restart $service

        # 记录日志
        logger "Restarted service: $service"
    fi
}

# 检查各个服务
check_service "backend" "http://localhost:8087/api/"
check_service "frontend" "http://localhost:8086/"
```

添加定时任务：

```bash
sudo crontab -e
```

添加以下行：

```
# 每 5 分钟检查一次服务状态
*/5 * * * * /usr/local/bin/lobster-monitor.sh
```

---

## 🔍 故障排查

### 问题 1：容器启动失败

**症状：**
```bash
docker-compose ps
# NAME                    STATUS
# lobster-postgres        Exit 1
# lobster-backend         Exit 1
```

**排查步骤：**

1. 查看容器日志
```bash
docker-compose logs postgres
docker-compose logs backend
```

2. 检查端口占用
```bash
sudo netstat -tulpn | grep -E '8086|8087|5432'
```

3. 检查磁盘空间
```bash
df -h
```

4. 重新构建镜像
```bash
docker-compose build --no-cache
docker-compose up -d
```

### 问题 2：数据库连接失败

**症状：**
```
django.db.utils.OperationalError: could not connect to server
```

**排查步骤：**

1. 检查数据库容器状态
```bash
docker-compose ps postgres
```

2. 查看数据库日志
```bash
docker-compose logs postgres
```

3. 测试数据库连接
```bash
docker-compose exec postgres psql -U postgres -d lobster_memory -c "SELECT version();"
```

4. 检查环境变量
```bash
docker-compose exec backend env | grep DB_
```

### 问题 3：前端无法访问后端 API

**症状：**
```
Network Error
```

**排查步骤：**

1. 检查后端服务状态
```bash
curl http://localhost:8087/api/
```

2. 检查前端配置
```bash
docker-compose logs frontend
```

3. 验证环境变量
```bash
docker-compose exec frontend env | grep REACT_APP_API_URL
```

4. 检查网络连接
```bash
docker-compose exec frontend curl http://backend:8087/api/
```

### 问题 4：文件扫描失败

**症状：**
```
Error reading /app/memory_files/file.txt: Permission denied
```

**排查步骤：**

1. 检查龙虾记忆目录挂载
```bash
docker-compose exec backend ls -la /app/memory_files
```

2. 检查目录权限
```bash
ls -ld /home/node/.openclaw/workspace/daotong
```

3. 修复权限
```bash
chmod -R 755 /home/node/.openclaw/workspace/daotong
```

4. 重新挂载
```bash
docker-compose down
docker-compose up -d
```

### 问题 5：内存占用过高

**症状：**
```
Memory usage exceeds 256MB
```

**排查步骤：**

1. 查看容器资源使用
```bash
docker stats
```

2. 检查大文件
```bash
find /home/node/.openclaw/workspace/daotong -type f -size +50M
```

3. 添加到 .lobsterignore
```bash
echo "*.log" >> /home/node/.openclaw/workspace/daotong/.lobsterignore
```

4. 重启服务
```bash
docker-compose restart backend
```

---

## 💾 备份与恢复

### 备份数据库

```bash
# 手动备份
docker-compose exec postgres pg_dump -U postgres lobster_memory > backup.sql

# 压缩备份
docker-compose exec postgres pg_dump -U postgres lobster_memory | gzip > backup.sql.gz

# 备份到远程
docker-compose exec postgres pg_dump -U postgres lobster_memory | gzip | ssh user@remote "cat > /backup/lobster_memory_$(date +%Y%m%d).sql.gz"
```

### 恢复数据库

```bash
# 从本地恢复
docker-compose exec -T postgres psql -U postgres lobster_memory < backup.sql

# 从压缩文件恢复
gunzip < backup.sql.gz | docker-compose exec -T postgres psql -U postgres lobster_memory

# 从远程恢复
ssh user@remote "cat /backup/lobster_memory_20240101.sql.gz" | gunzip | docker-compose exec -T postgres psql -U postgres lobster_memory
```

### 备份配置文件

```bash
# 备份配置文件
tar -czf lobster-config-$(date +%Y%m%d).tar.gz \
  .env \
  docker-compose.yml \
  .lobsterignore \
  nginx.conf
```

---

## ⚡ 性能优化

### 1. 数据库优化

```sql
-- 调整 PostgreSQL 配置
ALTER SYSTEM SET shared_buffers = '256MB';
ALTER SYSTEM SET effective_cache_size = '1GB';
ALTER SYSTEM SET maintenance_work_mem = '64MB';
ALTER SYSTEM SET work_mem = '16MB';

-- 重启数据库生效
docker-compose restart postgres
```

### 2. Nginx 优化

```nginx
# 启用 gzip 压缩
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml+rss application/json;

# 启用缓存
location ~* \.(jpg|jpeg|png|gif|ico|css|js)$ {
    expires 30d;
    add_header Cache-Control "public, immutable";
}
```

### 3. Docker 优化

```yaml
# docker-compose.yml
services:
  backend:
    # 限制内存使用
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 512M
        reservations:
          cpus: '0.5'
          memory: 256M
```

---

## 🔒 安全加固

### 1. 更新默认密码

```bash
# 修改数据库密码
docker-compose exec postgres psql -U postgres -c "ALTER USER postgres WITH PASSWORD 'your-strong-password';"

# 更新 .env 文件
sed -i 's/DB_PASSWORD=.*/DB_PASSWORD=your-strong-password/' .env
```

### 2. 配置防火墙

```bash
# 只允许特定 IP 访问数据库
sudo ufw allow from 192.168.1.0/24 to any port 5432
```

### 3. 启用 SELinux（CentOS）

```bash
# 检查 SELinux 状态
sestatus

# 启用 SELinux
sudo setenforce 1
```

### 4. 定期更新

```bash
# 更新 Docker 镜像
docker-compose pull
docker-compose up -d

# 更新系统包
sudo apt-get update && sudo apt-get upgrade -y
```

---

## 📝 维护清单

### 每日

- [ ] 检查服务状态
- [ ] 查看错误日志
- [ ] 验证备份完成

### 每周

- [ ] 清理旧日志
- [ ] 检查磁盘空间
- [ ] 更新系统补丁

### 每月

- [ ] 检查备份完整性
- [ ] 性能评估
- [ ] 安全审计

---

## 🆘 获取帮助

遇到问题？

1. 查看日志：`docker-compose logs -f`
2. 查看文档：[README.md](README.md)
3. 提交 Issue：http://10.2.0.100:8989/daotong/lobster-memory-sync/issues

---

**文档版本**: v1.0.0
**最后更新**: 2026-04-05
**维护者**: 道童