Gordon
2 years ago
commit
5b6c7479b4
@ -1,212 +1,504 @@
|
||||
# OpenIM enviroment
|
||||
|
||||
|
||||
## How to change the configuration
|
||||
|
||||
|
||||
**Modify the configuration files:**
|
||||
|
||||
Three ways to modify the configuration:
|
||||
|
||||
#### **1. Recommended using environment variables:**
|
||||
<!-- vscode-markdown-toc -->
|
||||
|
||||
* 1. [OpenIM Deployment Guide](#OpenIMDeploymentGuide)
|
||||
* 1.1. [Deployment Strategies](#DeploymentStrategies)
|
||||
* 1.2. [Source Code Deployment](#SourceCodeDeployment)
|
||||
* 1.3. [Docker Compose Deployment](#DockerComposeDeployment)
|
||||
* 1.4. [Environment Variable Configuration](#EnvironmentVariableConfiguration)
|
||||
* 1.4.1. [[1. Recommended using environment variables:](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#1-recommended-using-environment-variables)](#1.Recommendedusingenvironmentvariables:https:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.md1-recommended-using-environment-variables)
|
||||
* 1.4.2. [[Additional Configuration](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#additional-configuration)](#AdditionalConfigurationhttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdadditional-configuration)
|
||||
* 1.4.3. [[Security Considerations](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#security-considerations)](#SecurityConsiderationshttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdsecurity-considerations)
|
||||
* 1.4.4. [[Data Management](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#data-management)](#DataManagementhttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mddata-management)
|
||||
* 1.4.5. [[Monitoring and Logging](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#monitoring-and-logging)](#MonitoringandLogginghttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdmonitoring-and-logging)
|
||||
* 1.4.6. [[Troubleshooting](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#troubleshooting)](#Troubleshootinghttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdtroubleshooting)
|
||||
* 1.4.7. [[Conclusion](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#conclusion)](#Conclusionhttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdconclusion)
|
||||
* 1.4.8. [[Additional Resources](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#additional-resources)](#AdditionalResourceshttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdadditional-resources)
|
||||
* 2. [Further Configuration](#FurtherConfiguration)
|
||||
* 2.1. [Image Registry Configuration](#ImageRegistryConfiguration)
|
||||
* 2.2. [OpenIM Docker Network Configuration](#OpenIMDockerNetworkConfiguration)
|
||||
* 2.3. [OpenIM Configuration](#OpenIMConfiguration)
|
||||
* 2.4. [OpenIM Chat Configuration](#OpenIMChatConfiguration)
|
||||
* 2.5. [Zookeeper Configuration](#ZookeeperConfiguration)
|
||||
* 2.6. [MySQL Configuration](#MySQLConfiguration)
|
||||
* 2.7. [MongoDB Configuration](#MongoDBConfiguration)
|
||||
* 2.8. [Tencent Cloud COS Configuration](#TencentCloudCOSConfiguration)
|
||||
* 2.9. [Alibaba Cloud OSS Configuration](#AlibabaCloudOSSConfiguration)
|
||||
* 2.10. [Redis Configuration](#RedisConfiguration)
|
||||
* 2.11. [Kafka Configuration](#KafkaConfiguration)
|
||||
* 2.12. [OpenIM Web Configuration](#OpenIMWebConfiguration)
|
||||
* 2.13. [RPC Configuration](#RPCConfiguration)
|
||||
* 2.14. [Prometheus Configuration](#PrometheusConfiguration)
|
||||
* 2.15. [Grafana Configuration](#GrafanaConfiguration)
|
||||
* 2.16. [RPC Port Configuration Variables](#RPCPortConfigurationVariables)
|
||||
* 2.17. [RPC Register Name Configuration](#RPCRegisterNameConfiguration)
|
||||
* 2.18. [Log Configuration](#LogConfiguration)
|
||||
* 2.19. [Additional Configuration Variables](#AdditionalConfigurationVariables)
|
||||
* 2.20. [Prometheus Configuration](#PrometheusConfiguration-1)
|
||||
* 2.20.1. [General Configuration](#GeneralConfiguration)
|
||||
* 2.20.2. [Service-Specific Prometheus Ports](#Service-SpecificPrometheusPorts)
|
||||
|
||||
<!-- vscode-markdown-toc-config
|
||||
numbering=true
|
||||
autoSave=true
|
||||
/vscode-markdown-toc-config -->
|
||||
<!-- /vscode-markdown-toc -->
|
||||
|
||||
|
||||
## 1. <a name='OpenIMDeploymentGuide'></a>OpenIM Deployment Guide
|
||||
|
||||
Welcome to the OpenIM Deployment Guide! OpenIM offers a versatile and robust instant messaging server, and deploying it can be achieved through various methods. This guide will walk you through the primary deployment strategies, ensuring you can set up OpenIM in a way that best suits your needs.
|
||||
|
||||
### 1.1. <a name='DeploymentStrategies'></a>Deployment Strategies
|
||||
|
||||
OpenIM provides multiple deployment methods, each tailored to different use cases and technical preferences:
|
||||
|
||||
1. **[Source Code Deployment Guide](https://doc.rentsoft.cn/guides/gettingStarted/imSourceCodeDeployment)**
|
||||
2. **[Docker Deployment Guide](https://doc.rentsoft.cn/guides/gettingStarted/dockerCompose)**
|
||||
3. **[Kubernetes Deployment Guide](https://github.com/openimsdk/open-im-server/tree/main/deployments)**
|
||||
|
||||
While the first two methods will be our main focus, it's worth noting that the third method, Kubernetes deployment, is also viable and can be rendered via the `environment.sh` script variables.
|
||||
|
||||
### 1.2. <a name='SourceCodeDeployment'></a>Source Code Deployment
|
||||
|
||||
In the source code deployment method, the configuration generation process involves executing `make init`, which fundamentally runs the script `./scripts/init-config.sh`. This script utilizes variables defined in the [`environment.sh`](https://github.com/openimsdk/open-im-server/blob/main/scripts/install/environment.sh) script to render the [`openim.yaml`](https://github.com/openimsdk/open-im-server/blob/main/deployments/templates/openim.yaml) template file, subsequently generating the [`config.yaml`](https://github.com/openimsdk/open-im-server/blob/main/config/config.yaml) configuration file.
|
||||
|
||||
### 1.3. <a name='DockerComposeDeployment'></a>Docker Compose Deployment
|
||||
|
||||
Docker deployment offers a slightly more intricate template. Within the [openim-server](https://github.com/openimsdk/openim-docker/tree/main/openim-server) directory, multiple subdirectories correspond to various versions, each aligning with `openim-chat` as illustrated below:
|
||||
|
||||
| openim-server | openim-chat |
|
||||
| ------------------------------------------------------------ | ------------------------------------------------------------ |
|
||||
| [main](https://github.com/openimsdk/openim-docker/tree/main/openim-server/main) | [main](https://github.com/openimsdk/openim-docker/tree/main/openim-chat/main) |
|
||||
| [release-v3.2](https://github.com/openimsdk/openim-docker/tree/main/openim-server/release-v3.3) | [release-v3.2](https://github.com/openimsdk/openim-docker/tree/main/openim-chat/release-v1.3) |
|
||||
| [release-v3.2](https://github.com/openimsdk/openim-docker/tree/main/openim-server/release-v3.2) | [release-v3.2](https://github.com/openimsdk/openim-docker/tree/main/openim-chat/release-v1.2) |
|
||||
|
||||
Configuration file modifications can be made by specifying corresponding environment variables, for instance:
|
||||
|
||||
```bash
|
||||
export PASSWORD="openIM123" # Set password
|
||||
export USER="root" # Set username
|
||||
# Choose chat version and server version https://github.com/openimsdk/open-im-server/blob/main/docs/conversions/images.md, eg: main, release-v*.*
|
||||
export CHAT_BRANCH="main"
|
||||
export SERVER_BRANCH="main"
|
||||
#... Other environment variables
|
||||
# MONGO_USERNAME: This sets the MongoDB username
|
||||
# MONGO_PASSWORD: Set the MongoDB password
|
||||
# MONGO_DATABASE: Sets the MongoDB database name
|
||||
# MINIO_ENDPOINT: set the MinIO service address
|
||||
# API_URL: under network environment, set OpenIM Server API address
|
||||
export API_URL="http://127.0.0.1:10002"
|
||||
```
|
||||
|
||||
Next, update the configuration using `make init`:
|
||||
These variables are stored within the [`environment.sh`](https://github.com/OpenIMSDK/openim-docker/blob/main/scripts/install/environment.sh) configuration:
|
||||
|
||||
```bash
|
||||
make init
|
||||
readonly CHAT_BRANCH=${CHAT_BRANCH:-'main'}
|
||||
readonly SERVER_BRANCH=${SERVER_BRANCH:-'main'}
|
||||
```
|
||||
|
||||
#### **2. Modify the automation script:**
|
||||
Setting a variable, e.g., `export CHAT_BRANCH="release-v1.3"`, will prioritize `CHAT_BRANCH="release-v1.3"` as the variable value. Ultimately, the chosen image version is determined, and rendering is achieved through `make init` (or `./scripts/init-config.sh`).
|
||||
|
||||
> Note: Direct modifications to the `config.yaml` file are also permissible without utilizing `make init`.
|
||||
|
||||
### 1.4. <a name='EnvironmentVariableConfiguration'></a>Environment Variable Configuration
|
||||
|
||||
For convenience, configuration through modifying environment variables is recommended:
|
||||
|
||||
#### 1.4.1. <a name='1.Recommendedusingenvironmentvariables:https:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.md1-recommended-using-environment-variables'></a>[1. Recommended using environment variables:](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#1-recommended-using-environment-variables)
|
||||
|
||||
+ PASSWORD
|
||||
|
||||
+ **Description**: Password for mysql, mongodb, redis, and minio.
|
||||
+ **Default**: `openIM123`
|
||||
+ Notes:
|
||||
+ Minimum password length: 8 characters.
|
||||
+ Special characters are not allowed.
|
||||
|
||||
```bash
|
||||
scripts/install/environment.sh
|
||||
export PASSWORD="openIM123"
|
||||
```
|
||||
|
||||
Next, update the configuration using `make init`:
|
||||
+ USER
|
||||
|
||||
+ **Description**: Username for mysql, mongodb, redis, and minio.
|
||||
+ **Default**: `root`
|
||||
|
||||
```bash
|
||||
make init
|
||||
export USER="root"
|
||||
```
|
||||
|
||||
#### 3. Modify `config.yaml` and `.env` files (but will be overwritten when using `make init` again).
|
||||
|
||||
The `config/config.yaml` file has detailed configuration instructions for the storage components.
|
||||
+ API_URL
|
||||
|
||||
+ **Description**: API address.
|
||||
+ **Note**: If the server has an external IP, it will be automatically obtained. For internal networks, set this variable to the IP serving internally.
|
||||
|
||||
The config file is available via [environment.sh](https://github.com/openimsdk/open-im-server/blob/main/scripts/install/environment.sh) configuration [openim.yaml](https://github.com/openimsdk/open-im-server/blob/main/deployments/templates/openim.yaml) template, and then through the `make init` to automatically generate a new configuration.
|
||||
```
|
||||
export API_URL="http://ip:10002"
|
||||
```
|
||||
|
||||
+ DATA_DIR
|
||||
|
||||
## Environment variable
|
||||
+ **Description**: Data mount directory for components.
|
||||
+ **Default**: `/data/openim`
|
||||
|
||||
By setting the environment variable below, You can then refresh the configuration using `make init` or `./scripts/init-config.sh`
|
||||
```bash
|
||||
export DATA_DIR="/data/openim"
|
||||
```
|
||||
|
||||
##### MINIO
|
||||
#### 1.4.2. <a name='AdditionalConfigurationhttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdadditional-configuration'></a>[Additional Configuration](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#additional-configuration)
|
||||
|
||||
+ [MINIO DOCS](https://min.io/docs/minio/kubernetes/upstream/index.html)
|
||||
##### [MinIO Access and Secret Key](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#minio-access-and-secret-key)
|
||||
|
||||
apiURL is the address of the api, the access address of the app, use s3 must be configured
|
||||
To secure your MinIO server, you should set up an access key and secret key. These credentials are used to authenticate requests to your MinIO server.
|
||||
|
||||
#### Overview
|
||||
```bash
|
||||
export MINIO_ACCESS_KEY="YourAccessKey"
|
||||
export MINIO_SECRET_KEY="YourSecretKey"
|
||||
```
|
||||
|
||||
MinIO is an object storage server that is API compatible with Amazon S3. It's best suited for storing unstructured data such as photos, videos, log files, backups, and container/VM images. In this guide, we'll walk through the process of configuring MinIO with custom settings.
|
||||
##### [MinIO Browser](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#minio-browser)
|
||||
|
||||
#### Default Configuration
|
||||
MinIO comes with an embedded web-based object browser. You can control the availability of the MinIO browser by setting the `MINIO_BROWSER` environment variable.
|
||||
|
||||
Configuration can be achieved by modifying the default variables in the `./scripts/install/environment.sh` file. However, for more flexibility and dynamic adjustments, setting environment variables is recommended.
|
||||
```bash
|
||||
export MINIO_BROWSER="on"
|
||||
```
|
||||
|
||||
#### Setting Up the Environment Variables
|
||||
#### 1.4.3. <a name='SecurityConsiderationshttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdsecurity-considerations'></a>[Security Considerations](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#security-considerations)
|
||||
|
||||
##### IP Configuration
|
||||
##### [TLS/SSL Configuration](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#tls-ssl-configuration)
|
||||
|
||||
By default, the system generates the public IP of the machine. To manually set a public or local IP address, use:
|
||||
For secure communication, it's recommended to enable TLS/SSL for your MinIO server. You can do this by providing the path to the SSL certificate and key files.
|
||||
|
||||
```bash
|
||||
export IP=127.0.0.1
|
||||
export MINIO_CERTS_DIR="/path/to/certs/directory"
|
||||
```
|
||||
|
||||
##### API URL
|
||||
#### 1.4.4. <a name='DataManagementhttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mddata-management'></a>[Data Management](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#data-management)
|
||||
|
||||
##### [Data Retention Policy](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#data-retention-policy)
|
||||
|
||||
This is the address your application uses to communicate with MinIO. By default, it uses the public IP. However, you can adjust it to a public domain or another IP.
|
||||
You may want to set up a data retention policy to automatically delete objects after a specified period.
|
||||
|
||||
```bash
|
||||
export API_URL=127.0.0.1:10002
|
||||
export MINIO_RETENTION_DAYS="30"
|
||||
```
|
||||
|
||||
##### MinIO Endpoint Configuration
|
||||
#### 1.4.5. <a name='MonitoringandLogginghttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdmonitoring-and-logging'></a>[Monitoring and Logging](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#monitoring-and-logging)
|
||||
|
||||
##### [Audit Logging](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#audit-logging)
|
||||
|
||||
This is the primary address MinIO uses for communications:
|
||||
Enable audit logging to keep track of access and changes to your data.
|
||||
|
||||
```bash
|
||||
export MINIO_ENDPOINT="127.0.0.1"
|
||||
export MINIO_AUDIT="on"
|
||||
```
|
||||
|
||||
##### MinIO Sign Endpoint
|
||||
#### 1.4.6. <a name='Troubleshootinghttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdtroubleshooting'></a>[Troubleshooting](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#troubleshooting)
|
||||
|
||||
For direct external access to stored content:
|
||||
##### [Debug Mode](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#debug-mode)
|
||||
|
||||
In case of issues, you may enable debug mode to get more detailed logs to assist in troubleshooting.
|
||||
|
||||
```bash
|
||||
export MINIO_SIGN_ENDPOINT=127.0.0.1:10005
|
||||
export MINIO_DEBUG="on"
|
||||
```
|
||||
|
||||
##### Modifying MinIO's Port
|
||||
#### 1.4.7. <a name='Conclusionhttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdconclusion'></a>[Conclusion](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#conclusion)
|
||||
|
||||
If you need to adjust MinIO's port from the default:
|
||||
With the environment variables configured as per your requirements, your MinIO server should be ready to securely store and manage your object data. Ensure to verify the setup and monitor the logs for any unusual activities or errors. Regularly update the MinIO server and review your configuration to adapt to any changes or improvements in the MinIO system.
|
||||
|
||||
```bash
|
||||
export MINIO_PORT="10005"
|
||||
```
|
||||
#### 1.4.8. <a name='AdditionalResourceshttps:github.comopenimsdkopen-im-serverblobmaindocscontribenvironment.mdadditional-resources'></a>[Additional Resources](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/environment.md#additional-resources)
|
||||
|
||||
#### Applying the Configuration
|
||||
+ [MinIO Client Quickstart Guide](https://docs.min.io/docs/minio-client-quickstart-guide)
|
||||
+ [MinIO Admin Complete Guide](https://docs.min.io/docs/minio-admin-complete-guide)
|
||||
+ [MinIO Docker Quickstart Guide](https://docs.min.io/docs/minio-docker-quickstart-guide)
|
||||
|
||||
After setting your desired environment variables, restart the MinIO server to apply the changes.
|
||||
Feel free to explore the MinIO documentation for more advanced configurations and usage scenarios.
|
||||
|
||||
#### Verification
|
||||
|
||||
It's crucial to verify the configurations by checking the connectivity between your application and MinIO using the set API URL and ensuring that the data can be directly accessed using the `signEndpoint`.
|
||||
|
||||
## 2. <a name='FurtherConfiguration'></a>Further Configuration
|
||||
|
||||
## Configuration Details
|
||||
### 2.1. <a name='ImageRegistryConfiguration'></a>Image Registry Configuration
|
||||
|
||||
###### Zookeeper
|
||||
**Description**: The image registry configuration allows users to select an image address for use. The default is set to use GITHUB images, but users can opt for Docker Hub or Ali Cloud, especially beneficial for Chinese users due to its local proximity.
|
||||
|
||||
- **Purpose**: Used for RPC service discovery and registration, cluster support.
|
||||
| Parameter | Default Value | Description |
|
||||
| ---------------- | --------------------- | ------------------------------------------------------------ |
|
||||
| `IMAGE_REGISTRY` | `"ghcr.io/openimsdk"` | The registry from which Docker images will be pulled. Other options include `"openim"` and `"registry.cn-hangzhou.aliyuncs.com/openimsdk"`. |
|
||||
|
||||
```bash
|
||||
zookeeper:
|
||||
schema: openim # Not recommended to modify
|
||||
address: [ 127.0.0.1:2181 ] # Address
|
||||
username: # Username
|
||||
password: # Password
|
||||
```
|
||||
### 2.2. <a name='OpenIMDockerNetworkConfiguration'></a>OpenIM Docker Network Configuration
|
||||
|
||||
###### MySQL
|
||||
**Description**: This section configures the Docker network subnet and generates IP addresses for various services within the defined subnet.
|
||||
|
||||
- **Purpose**: Used for storing users, relationships, and groups. Supports master-slave database.
|
||||
| Parameter | Example Value | Description |
|
||||
| --------------------------- | ----------------- | ------------------------------------------------------------ |
|
||||
| `DOCKER_BRIDGE_SUBNET` | `'172.28.0.0/16'` | The subnet for the Docker network. |
|
||||
| `DOCKER_BRIDGE_GATEWAY` | Generated IP | The gateway IP address within the Docker subnet. |
|
||||
| `[SERVICE]_NETWORK_ADDRESS` | Generated IP | The network IP address for a specific service (e.g., MYSQL, MONGO, REDIS, etc.) within the Docker subnet. |
|
||||
|
||||
```bash
|
||||
mysql:
|
||||
address: [ 127.0.0.1:13306 ] # Address
|
||||
username: root # Username
|
||||
password: openIM123 # Password
|
||||
database: openIM_v2 # Not recommended to modify
|
||||
maxOpenConn: 1000 # Maximum connection
|
||||
maxIdleConn: 100 # Maximum idle connection
|
||||
maxLifeTime: 60 # Max time a connection can be reused (seconds)
|
||||
logLevel: 4 # Log level (1=silent, 2=error, 3=warn, 4=info)
|
||||
slowThreshold: 500 # Slow statement threshold (milliseconds)
|
||||
```
|
||||
### 2.3. <a name='OpenIMConfiguration'></a>OpenIM Configuration
|
||||
|
||||
###### Mongo
|
||||
**Description**: OpenIM configuration involves setting up directories for data, installation, configuration, and logs. It also involves configuring the OpenIM server address and ports for WebSocket and API.
|
||||
|
||||
- **Purpose**: Used for storing offline messages. Supports mongo sharded clusters.
|
||||
| Parameter | Default Value | Description |
|
||||
| ----------------------- | ------------------------ | ----------------------------------------- |
|
||||
| `OPENIM_DATA_DIR` | `"/data/openim"` | Directory for OpenIM data. |
|
||||
| `OPENIM_INSTALL_DIR` | `"/opt/openim"` | Directory where OpenIM is installed. |
|
||||
| `OPENIM_CONFIG_DIR` | `"/etc/openim"` | Directory for OpenIM configuration files. |
|
||||
| `OPENIM_LOG_DIR` | `"/var/log/openim"` | Directory for OpenIM logs. |
|
||||
| `OPENIM_SERVER_ADDRESS` | Docker Bridge Gateway IP | OpenIM server address. |
|
||||
| `OPENIM_WS_PORT` | `'10001'` | Port for OpenIM WebSocket. |
|
||||
| `API_OPENIM_PORT` | `'10002'` | Port for OpenIM API. |
|
||||
|
||||
```bash
|
||||
mongo:
|
||||
uri: # Use this value directly if not empty
|
||||
address: [ 127.0.0.1:37017 ] # Address
|
||||
database: openIM # Default mongo db
|
||||
username: root # Username
|
||||
password: openIM123 # Password
|
||||
maxPoolSize: 100 # Maximum connections
|
||||
```
|
||||
### 2.4. <a name='OpenIMChatConfiguration'></a>OpenIM Chat Configuration
|
||||
|
||||
###### Redis
|
||||
**Description**: Configuration for OpenIM chat, including data directory, server address, and ports for API and chat functionalities.
|
||||
|
||||
- **Purpose**: Used for storing message sequence numbers, latest messages, user tokens, and MySQL cache. Supports cluster deployment.
|
||||
| Parameter | Example Value | Description |
|
||||
| ----------------------- | -------------------------- | ------------------------------- |
|
||||
| `OPENIM_CHAT_DATA_DIR` | `"./openim-chat/[BRANCH]"` | Directory for OpenIM chat data. |
|
||||
| `OPENIM_CHAT_ADDRESS` | Docker Bridge Gateway IP | OpenIM chat service address. |
|
||||
| `OPENIM_CHAT_API_PORT` | `"10008"` | Port for OpenIM chat API. |
|
||||
| `OPENIM_ADMIN_API_PORT` | `"10009"` | Port for OpenIM Admin API. |
|
||||
| `OPENIM_ADMIN_PORT` | `"30200"` | Port for OpenIM chat Admin. |
|
||||
| `OPENIM_CHAT_PORT` | `"30300"` | Port for OpenIM chat. |
|
||||
|
||||
```bash
|
||||
redis:
|
||||
address: [ 127.0.0.1:16379 ] # Address
|
||||
username: # Username
|
||||
password: openIM123 # Password
|
||||
```
|
||||
### 2.5. <a name='ZookeeperConfiguration'></a>Zookeeper Configuration
|
||||
|
||||
###### Kafka
|
||||
**Description**: Configuration for Zookeeper, including schema, port, address, and credentials.
|
||||
|
||||
- **Purpose**: Used for message queues for decoupling. Supports cluster deployment.
|
||||
| Parameter | Example Value | Description |
|
||||
| -------------------- | ------------------------ | ----------------------- |
|
||||
| `ZOOKEEPER_SCHEMA` | `"openim"` | Schema for Zookeeper. |
|
||||
| `ZOOKEEPER_PORT` | `"12181"` | Port for Zookeeper. |
|
||||
| `ZOOKEEPER_ADDRESS` | Docker Bridge Gateway IP | Address for Zookeeper. |
|
||||
| `ZOOKEEPER_USERNAME` | `""` | Username for Zookeeper. |
|
||||
| `ZOOKEEPER_PASSWORD` | `""` | Password for Zookeeper. |
|
||||
|
||||
```bash
|
||||
kafka:
|
||||
username: # Username
|
||||
password: # Password
|
||||
addr: [ 127.0.0.1:9092 ] # Address
|
||||
topics:
|
||||
latestMsgToRedis: "latestMsgToRedis"
|
||||
offlineMsgToMongo: "offlineMsgToMongoMysql"
|
||||
msgToPush: "msgToPush"
|
||||
msgToModify: "msgToModify"
|
||||
consumerGroupID:
|
||||
msgToRedis: redis
|
||||
msgToMongo: mongo
|
||||
msgToMySql: mysql
|
||||
msgToPush: push
|
||||
msgToModify: modify
|
||||
```
|
||||
### 2.6. <a name='MySQLConfiguration'></a>MySQL Configuration
|
||||
|
||||
**Description**: Configuration for MySQL, including port, address, and credentials.
|
||||
|
||||
## Config options
|
||||
| Parameter | Example Value | Description |
|
||||
| ---------------- | ------------------------ | ------------------- |
|
||||
| `MYSQL_PORT` | `"13306"` | Port for MySQL. |
|
||||
| `MYSQL_ADDRESS` | Docker Bridge Gateway IP | Address for MySQL. |
|
||||
| `MYSQL_USERNAME` | User-defined | Username for MySQL. |
|
||||
| `MYSQL_PASSWORD` | User-defined | Password for MySQL. |
|
||||
|
||||
...
|
||||
Note: The configurations for other services (e.g., MONGO, REDIS, KAFKA, etc.) follow a similar pattern to MySQL and can be documented in a similar manner.
|
||||
|
||||
## Use the default values
|
||||
### 2.7. <a name='MongoDBConfiguration'></a>MongoDB Configuration
|
||||
|
||||
A method to revert to the default value:
|
||||
This section involves setting up MongoDB, including its port, address, and credentials.
|
||||
|
||||
```bash
|
||||
export IP=127.0.0.1
|
||||
```
|
||||
| Parameter | Example Value | Description |
|
||||
| -------------- | -------------- | ----------------------- |
|
||||
| MONGO_PORT | "27017" | Port used by MongoDB. |
|
||||
| MONGO_ADDRESS | [Generated IP] | IP address for MongoDB. |
|
||||
| MONGO_USERNAME | [User Defined] | Username for MongoDB. |
|
||||
| MONGO_PASSWORD | [User Defined] | Password for MongoDB. |
|
||||
|
||||
### 2.8. <a name='TencentCloudCOSConfiguration'></a>Tencent Cloud COS Configuration
|
||||
|
||||
This section involves setting up Tencent Cloud COS, including its bucket URL and credentials.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| ----------------- | ------------------------------------------------------------ | ------------------------------------ |
|
||||
| COS_BUCKET_URL | "[https://temp-1252357374.cos.ap-chengdu.myqcloud.com](https://temp-1252357374.cos.ap-chengdu.myqcloud.com/)" | Tencent Cloud COS bucket URL. |
|
||||
| COS_SECRET_ID | [User Defined] | Secret ID for Tencent Cloud COS. |
|
||||
| COS_SECRET_KEY | [User Defined] | Secret key for Tencent Cloud COS. |
|
||||
| COS_SESSION_TOKEN | [User Defined] | Session token for Tencent Cloud COS. |
|
||||
| COS_PUBLIC_READ | "false" | Public read access. |
|
||||
|
||||
### 2.9. <a name='AlibabaCloudOSSConfiguration'></a>Alibaba Cloud OSS Configuration
|
||||
|
||||
This section involves setting up Alibaba Cloud OSS, including its endpoint, bucket name, and credentials.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| --------------------- | ------------------------------------------------------------ | ---------------------------------------- |
|
||||
| OSS_ENDPOINT | "[https://oss-cn-chengdu.aliyuncs.com](https://oss-cn-chengdu.aliyuncs.com/)" | Endpoint URL for Alibaba Cloud OSS. |
|
||||
| OSS_BUCKET | "demo-9999999" | Bucket name for Alibaba Cloud OSS. |
|
||||
| OSS_BUCKET_URL | "[https://demo-9999999.oss-cn-chengdu.aliyuncs.com](https://demo-9999999.oss-cn-chengdu.aliyuncs.com/)" | Bucket URL for Alibaba Cloud OSS. |
|
||||
| OSS_ACCESS_KEY_ID | [User Defined] | Access key ID for Alibaba Cloud OSS. |
|
||||
| OSS_ACCESS_KEY_SECRET | [User Defined] | Access key secret for Alibaba Cloud OSS. |
|
||||
| OSS_SESSION_TOKEN | [User Defined] | Session token for Alibaba Cloud OSS. |
|
||||
| OSS_PUBLIC_READ | "false" | Public read access. |
|
||||
|
||||
### 2.10. <a name='RedisConfiguration'></a>Redis Configuration
|
||||
|
||||
This section involves setting up Redis, including its port, address, and credentials.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| -------------- | -------------------------- | --------------------- |
|
||||
| REDIS_PORT | "16379" | Port used by Redis. |
|
||||
| REDIS_ADDRESS | "${DOCKER_BRIDGE_GATEWAY}" | IP address for Redis. |
|
||||
| REDIS_USERNAME | [User Defined] | Username for Redis. |
|
||||
| REDIS_PASSWORD | "${PASSWORD}" | Password for Redis. |
|
||||
|
||||
### 2.11. <a name='KafkaConfiguration'></a>Kafka Configuration
|
||||
|
||||
This section involves setting up Kafka, including its port, address, credentials, and topics.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| ---------------------------- | -------------------------- | ----------------------------------- |
|
||||
| KAFKA_USERNAME | [User Defined] | Username for Kafka. |
|
||||
| KAFKA_PASSWORD | [User Defined] | Password for Kafka. |
|
||||
| KAFKA_PORT | "19094" | Port used by Kafka. |
|
||||
| KAFKA_ADDRESS | "${DOCKER_BRIDGE_GATEWAY}" | IP address for Kafka. |
|
||||
| KAFKA_LATESTMSG_REDIS_TOPIC | "latestMsgToRedis" | Topic for latest message to Redis. |
|
||||
| KAFKA_OFFLINEMSG_MONGO_TOPIC | "offlineMsgToMongoMysql" | Topic for offline message to Mongo. |
|
||||
| KAFKA_MSG_PUSH_TOPIC | "msgToPush" | Topic for message to push. |
|
||||
| KAFKA_CONSUMERGROUPID_REDIS | "redis" | Consumer group ID to Redis. |
|
||||
| KAFKA_CONSUMERGROUPID_MONGO | "mongo" | Consumer group ID to Mongo. |
|
||||
| KAFKA_CONSUMERGROUPID_MYSQL | "mysql" | Consumer group ID to MySQL. |
|
||||
| KAFKA_CONSUMERGROUPID_PUSH | "push" | Consumer group ID to push. |
|
||||
|
||||
Note: Ensure to replace placeholder values (like [User Defined], `${DOCKER_BRIDGE_GATEWAY}`, and `${PASSWORD}`) with actual values before deploying the configuration.
|
||||
|
||||
|
||||
|
||||
### 2.12. <a name='OpenIMWebConfiguration'></a>OpenIM Web Configuration
|
||||
|
||||
This section involves setting up OpenIM Web, including its port, address, and dist path.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| -------------------- | -------------------------- | ------------------------- |
|
||||
| OPENIM_WEB_PORT | "11001" | Port used by OpenIM Web. |
|
||||
| OPENIM_WEB_ADDRESS | "${DOCKER_BRIDGE_GATEWAY}" | Address for OpenIM Web. |
|
||||
| OPENIM_WEB_DIST_PATH | "/app/dist" | Dist path for OpenIM Web. |
|
||||
|
||||
### 2.13. <a name='RPCConfiguration'></a>RPC Configuration
|
||||
|
||||
Configuration for RPC, including the register and listen IP.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| --------------- | -------------- | -------------------- |
|
||||
| RPC_REGISTER_IP | [User Defined] | Register IP for RPC. |
|
||||
| RPC_LISTEN_IP | "0.0.0.0" | Listen IP for RPC. |
|
||||
|
||||
### 2.14. <a name='PrometheusConfiguration'></a>Prometheus Configuration
|
||||
|
||||
Setting up Prometheus, including its port and address.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| ------------------ | -------------------------- | ------------------------ |
|
||||
| PROMETHEUS_PORT | "19090" | Port used by Prometheus. |
|
||||
| PROMETHEUS_ADDRESS | "${DOCKER_BRIDGE_GATEWAY}" | Address for Prometheus. |
|
||||
|
||||
### 2.15. <a name='GrafanaConfiguration'></a>Grafana Configuration
|
||||
|
||||
Configuration for Grafana, including its port and address.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| --------------- | -------------------------- | --------------------- |
|
||||
| GRAFANA_PORT | "3000" | Port used by Grafana. |
|
||||
| GRAFANA_ADDRESS | "${DOCKER_BRIDGE_GATEWAY}" | Address for Grafana. |
|
||||
|
||||
### 2.16. <a name='RPCPortConfigurationVariables'></a>RPC Port Configuration Variables
|
||||
|
||||
Configuration for various RPC ports. Note: For launching multiple programs, just fill in multiple ports separated by commas. Try not to have spaces.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| --------------------------- | ------------- | ----------------------------------- |
|
||||
| OPENIM_USER_PORT | '10110' | OpenIM User Service Port. |
|
||||
| OPENIM_FRIEND_PORT | '10120' | OpenIM Friend Service Port. |
|
||||
| OPENIM_MESSAGE_PORT | '10130' | OpenIM Message Service Port. |
|
||||
| OPENIM_MESSAGE_GATEWAY_PORT | '10140' | OpenIM Message Gateway Service Port |
|
||||
| OPENIM_GROUP_PORT | '10150' | OpenIM Group Service Port. |
|
||||
| OPENIM_AUTH_PORT | '10160' | OpenIM Authorization Service Port. |
|
||||
| OPENIM_PUSH_PORT | '10170' | OpenIM Push Service Port. |
|
||||
| OPENIM_CONVERSATION_PORT | '10180' | OpenIM Conversation Service Port. |
|
||||
| OPENIM_THIRD_PORT | '10190' | OpenIM Third-Party Service Port. |
|
||||
|
||||
### 2.17. <a name='RPCRegisterNameConfiguration'></a>RPC Register Name Configuration
|
||||
|
||||
This section involves setting up the RPC Register Names for various OpenIM services.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| --------------------------- | ---------------- | ----------------------------------- |
|
||||
| OPENIM_USER_NAME | "User" | OpenIM User Service Name |
|
||||
| OPENIM_FRIEND_NAME | "Friend" | OpenIM Friend Service Name |
|
||||
| OPENIM_MSG_NAME | "Msg" | OpenIM Message Service Name |
|
||||
| OPENIM_PUSH_NAME | "Push" | OpenIM Push Service Name |
|
||||
| OPENIM_MESSAGE_GATEWAY_NAME | "MessageGateway" | OpenIM Message Gateway Service Name |
|
||||
| OPENIM_GROUP_NAME | "Group" | OpenIM Group Service Name |
|
||||
| OPENIM_AUTH_NAME | "Auth" | OpenIM Authorization Service Name |
|
||||
| OPENIM_CONVERSATION_NAME | "Conversation" | OpenIM Conversation Service Name |
|
||||
| OPENIM_THIRD_NAME | "Third" | OpenIM Third-Party Service Name |
|
||||
|
||||
### 2.18. <a name='LogConfiguration'></a>Log Configuration
|
||||
|
||||
This section involves configuring the log settings, including storage location, rotation time, and log level.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| ------------------------- | ------------------------ | --------------------------------- |
|
||||
| LOG_STORAGE_LOCATION | ""${OPENIM_ROOT}"/logs/" | Location for storing logs |
|
||||
| LOG_ROTATION_TIME | "24" | Log rotation time (in hours) |
|
||||
| LOG_REMAIN_ROTATION_COUNT | "2" | Number of log rotations to retain |
|
||||
| LOG_REMAIN_LOG_LEVEL | "6" | Log level to retain |
|
||||
| LOG_IS_STDOUT | "false" | Output log to standard output |
|
||||
| LOG_IS_JSON | "false" | Log in JSON format |
|
||||
| LOG_WITH_STACK | "false" | Include stack info in logs |
|
||||
|
||||
### 2.19. <a name='AdditionalConfigurationVariables'></a>Additional Configuration Variables
|
||||
|
||||
This section involves setting up additional configuration variables for Websocket, Push Notifications, and Chat.
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| ----------------------- | ----------------- | ---------------------------------- |
|
||||
| WEBSOCKET_MAX_CONN_NUM | "100000" | Maximum Websocket connections |
|
||||
| WEBSOCKET_MAX_MSG_LEN | "4096" | Maximum Websocket message length |
|
||||
| WEBSOCKET_TIMEOUT | "10" | Websocket timeout |
|
||||
| PUSH_ENABLE | "getui" | Push notification enable status |
|
||||
| GETUI_PUSH_URL | [Generated URL] | GeTui Push Notification URL |
|
||||
| GETUI_MASTER_SECRET | [User Defined] | GeTui Master Secret |
|
||||
| GETUI_APP_KEY | [User Defined] | GeTui Application Key |
|
||||
| GETUI_INTENT | [User Defined] | GeTui Push Intent |
|
||||
| GETUI_CHANNEL_ID | [User Defined] | GeTui Channel ID |
|
||||
| GETUI_CHANNEL_NAME | [User Defined] | GeTui Channel Name |
|
||||
| FCM_SERVICE_ACCOUNT | "x.json" | FCM Service Account |
|
||||
| JPNS_APP_KEY | [User Defined] | JPNS Application Key |
|
||||
| JPNS_MASTER_SECRET | [User Defined] | JPNS Master Secret |
|
||||
| JPNS_PUSH_URL | [User Defined] | JPNS Push Notification URL |
|
||||
| JPNS_PUSH_INTENT | [User Defined] | JPNS Push Intent |
|
||||
| MANAGER_USERID_1 | "openIM123456" | Administrator ID 1 |
|
||||
| MANAGER_USERID_2 | "openIM654321" | Administrator ID 2 |
|
||||
| MANAGER_USERID_3 | "openIMAdmin" | Administrator ID 3 |
|
||||
| NICKNAME_1 | "system1" | Nickname 1 |
|
||||
| NICKNAME_2 | "system2" | Nickname 2 |
|
||||
| NICKNAME_3 | "system3" | Nickname 3 |
|
||||
| MULTILOGIN_POLICY | "1" | Multi-login Policy |
|
||||
| CHAT_PERSISTENCE_MYSQL | "true" | Chat Persistence in MySQL |
|
||||
| MSG_CACHE_TIMEOUT | "86400" | Message Cache Timeout |
|
||||
| GROUP_MSG_READ_RECEIPT | "true" | Group Message Read Receipt Enable |
|
||||
| SINGLE_MSG_READ_RECEIPT | "true" | Single Message Read Receipt Enable |
|
||||
| RETAIN_CHAT_RECORDS | "365" | Retain Chat Records (in days) |
|
||||
| CHAT_RECORDS_CLEAR_TIME | [Cron Expression] | Chat Records Clear Time |
|
||||
| MSG_DESTRUCT_TIME | [Cron Expression] | Message Destruct Time |
|
||||
| SECRET | "${PASSWORD}" | Secret Key |
|
||||
| TOKEN_EXPIRE | "90" | Token Expiry Time |
|
||||
| FRIEND_VERIFY | "false" | Friend Verification Enable |
|
||||
| IOS_PUSH_SOUND | "xxx" | iOS |
|
||||
|
||||
|
||||
|
||||
### 2.20. <a name='PrometheusConfiguration-1'></a>Prometheus Configuration
|
||||
|
||||
This section involves configuring Prometheus, including enabling/disabling it and setting up ports for various services.
|
||||
|
||||
#### 2.20.1. <a name='GeneralConfiguration'></a>General Configuration
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| ------------------- | ------------- | ----------------------------- |
|
||||
| `PROMETHEUS_ENABLE` | "false" | Whether to enable Prometheus. |
|
||||
|
||||
#### 2.20.2. <a name='Service-SpecificPrometheusPorts'></a>Service-Specific Prometheus Ports
|
||||
|
||||
| Service | Parameter | Default Port Value | Description |
|
||||
| ------------------------ | ------------------------ | ---------------------------- | -------------------------------------------------- |
|
||||
| User Service | `USER_PROM_PORT` | '20110' | Prometheus port for the User service. |
|
||||
| Friend Service | `FRIEND_PROM_PORT` | '20120' | Prometheus port for the Friend service. |
|
||||
| Message Service | `MESSAGE_PROM_PORT` | '20130' | Prometheus port for the Message service. |
|
||||
| Message Gateway | `MSG_GATEWAY_PROM_PORT` | '20140' | Prometheus port for the Message Gateway. |
|
||||
| Group Service | `GROUP_PROM_PORT` | '20150' | Prometheus port for the Group service. |
|
||||
| Auth Service | `AUTH_PROM_PORT` | '20160' | Prometheus port for the Auth service. |
|
||||
| Push Service | `PUSH_PROM_PORT` | '20170' | Prometheus port for the Push service. |
|
||||
| Conversation Service | `CONVERSATION_PROM_PORT` | '20230' | Prometheus port for the Conversation service. |
|
||||
| RTC Service | `RTC_PROM_PORT` | '21300' | Prometheus port for the RTC service. |
|
||||
| Third Service | `THIRD_PROM_PORT` | '21301' | Prometheus port for the Third service. |
|
||||
| Message Transfer Service | `MSG_TRANSFER_PROM_PORT` | '21400, 21401, 21402, 21403' | Prometheus ports for the Message Transfer service. |
|
||||
|
||||
@ -0,0 +1,61 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/openimsdk/open-im-server/v3/tools/data-conversion/chat/conversion"
|
||||
"github.com/openimsdk/open-im-server/v3/tools/data-conversion/utils"
|
||||
"gorm.io/driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
"log"
|
||||
)
|
||||
|
||||
func main() {
|
||||
var (
|
||||
usernameV2 = "root" // v2版本mysql用户名
|
||||
passwordV2 = "openIM" // v2版本mysql密码
|
||||
addrV2 = "127.0.0.1:13306" // v2版本mysql地址
|
||||
databaseV2 = "admin_chat" // v2版本mysql数据库名字
|
||||
)
|
||||
|
||||
var (
|
||||
usernameV3 = "root" // v3版本mysql用户名
|
||||
passwordV3 = "openIM123" // v3版本mysql密码
|
||||
addrV3 = "127.0.0.1:13306" // v3版本mysql地址
|
||||
databaseV3 = "openim_enterprise" // v3版本mysql数据库名字
|
||||
)
|
||||
|
||||
var concurrency = 1 // 并发数量
|
||||
|
||||
log.SetFlags(log.LstdFlags | log.Llongfile)
|
||||
dsnV2 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", usernameV2, passwordV2, addrV2, databaseV2)
|
||||
dsnV3 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", usernameV3, passwordV3, addrV3, databaseV3)
|
||||
dbV2, err := gorm.Open(mysql.Open(dsnV2), &gorm.Config{Logger: logger.Discard})
|
||||
if err != nil {
|
||||
log.Println("open v2 db failed", err)
|
||||
return
|
||||
}
|
||||
dbV3, err := gorm.Open(mysql.Open(dsnV3), &gorm.Config{Logger: logger.Discard})
|
||||
if err != nil {
|
||||
log.Println("open v3 db failed", err)
|
||||
return
|
||||
}
|
||||
|
||||
var tasks utils.TakeList
|
||||
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Account) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Attribute) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Register) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.UserLoginRecord) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Admin) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Applet) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.ForbiddenAccount) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.InvitationRegister) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.IPForbidden) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.LimitUserLoginIP) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.RegisterAddFriend) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.RegisterAddGroup) })
|
||||
|
||||
utils.RunTask(concurrency, tasks)
|
||||
|
||||
}
|
||||
@ -1,33 +0,0 @@
|
||||
package conversion
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/schema"
|
||||
)
|
||||
|
||||
func FindAndInsert[V2 any, V3 schema.Tabler](v2db *gorm.DB, v3db *gorm.DB, fn func(V2) V3) (string, error) {
|
||||
var t V3
|
||||
name := t.TableName()
|
||||
if err := v3db.AutoMigrate(&t); err != nil {
|
||||
return name, fmt.Errorf("auto migrate v3 %s failed %w", name, err)
|
||||
}
|
||||
const size = 100
|
||||
for i := 0; ; i++ {
|
||||
var v2s []V2
|
||||
if err := v2db.Offset(i * size).Limit(size).Find(&v2s).Error; err != nil {
|
||||
return name, fmt.Errorf("find v2 %s failed %w", name, err)
|
||||
}
|
||||
if len(v2s) == 0 {
|
||||
return name, nil
|
||||
}
|
||||
v3s := make([]V3, 0, len(v2s))
|
||||
for _, v := range v2s {
|
||||
v3s = append(v3s, fn(v))
|
||||
}
|
||||
if err := v3db.Create(&v3s).Error; err != nil {
|
||||
return name, fmt.Errorf("insert v3 %s failed %w", name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,69 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
"gorm.io/driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"github.com/openimsdk/open-im-server/v3/tools/data-conversion/chat/conversion"
|
||||
)
|
||||
|
||||
func main() {
|
||||
var (
|
||||
usernameV2 = "root"
|
||||
passwordV2 = "openIM123"
|
||||
addrV2 = "127.0.0.1:13306"
|
||||
databaseV2 = "admin_chat"
|
||||
)
|
||||
|
||||
var (
|
||||
usernameV3 = "root"
|
||||
passwordV3 = "openIM123"
|
||||
addrV3 = "127.0.0.1:13306"
|
||||
databaseV3 = "openim_enterprise"
|
||||
)
|
||||
|
||||
dsnV2 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", usernameV2, passwordV2, addrV2, databaseV2)
|
||||
dsnV3 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", usernameV3, passwordV3, addrV3, databaseV3)
|
||||
dbV2, err := gorm.Open(mysql.Open(dsnV2), &gorm.Config{})
|
||||
if err != nil {
|
||||
log.Println("open v2 db failed", err)
|
||||
return
|
||||
}
|
||||
dbV3, err := gorm.Open(mysql.Open(dsnV3), &gorm.Config{})
|
||||
if err != nil {
|
||||
log.Println("open v3 db failed", err)
|
||||
return
|
||||
}
|
||||
|
||||
var fns []func() (string, error)
|
||||
|
||||
Append := func(fn func() (string, error)) {
|
||||
fns = append(fns, fn)
|
||||
}
|
||||
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.Account) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.Attribute) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.Register) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.UserLoginRecord) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.Admin) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.Applet) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.ForbiddenAccount) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.InvitationRegister) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.IPForbidden) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.LimitUserLoginIP) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.RegisterAddFriend) })
|
||||
Append(func() (string, error) { return conversion.FindAndInsert(dbV2, dbV3, conversion.RegisterAddGroup) })
|
||||
|
||||
for i := range fns {
|
||||
name, err := fns[i]()
|
||||
if err == nil {
|
||||
log.Printf("[%d/%d] %s success\n", i+1, len(fns), name)
|
||||
} else {
|
||||
log.Printf("[%d/%d] %s failed %s\n", i+1, len(fns), name, err)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,68 @@
|
||||
# v2数据迁移工具
|
||||
|
||||
### <font color=red>转换前请做好数据备份！！！</font>
|
||||
|
||||
### 转换OPENIM MYSQL数据
|
||||
- open-im-server/v3/tools/data-conversion/openim/mysql.go
|
||||
- 配置mysql.go数据库信息
|
||||
- 需要手动创建v3版本数据库,字符集`utf8mb4`
|
||||
|
||||
```go
|
||||
var (
|
||||
usernameV2 = "root" // v2版本mysql用户名
|
||||
passwordV2 = "openIM" // v2版本mysql密码
|
||||
addrV2 = "127.0.0.1:13306" // v2版本mysql地址
|
||||
databaseV2 = "openIM_v2" // v2版本mysql数据库名字
|
||||
)
|
||||
|
||||
var (
|
||||
usernameV3 = "root" // v3版本mysql用户名
|
||||
passwordV3 = "openIM123" // v3版本mysql密码
|
||||
addrV3 = "127.0.0.1:13306" // v3版本mysql地址
|
||||
databaseV3 = "openIM_v3" // v3版本mysql数据库名字
|
||||
)
|
||||
```
|
||||
```shell
|
||||
go run mysql.go
|
||||
```
|
||||
|
||||
### 转换聊天消息(可选)
|
||||
- 目前只支持转换kafka中的消息
|
||||
- open-im-server/v3/tools/data-conversion/openim/msg.go
|
||||
- 配置msg.go数据库信息
|
||||
```go
|
||||
var (
|
||||
topic = "ws2ms_chat" // v2版本配置文件kafka.topic.ws2ms_chat
|
||||
kafkaAddr = "127.0.0.1:9092" // v2版本配置文件kafka.topic.addr
|
||||
rpcAddr = "127.0.0.1:10130" // v3版本配置文件rpcPort.openImMessagePort
|
||||
adminUserID = "openIM123456" // v3版本管理员userID
|
||||
concurrency = 4 // 并发数量
|
||||
)
|
||||
```
|
||||
```shell
|
||||
go run msg.go
|
||||
```
|
||||
|
||||
### 转换业务服务器数据(使用官方业务服务器需要转换)
|
||||
- 目前只支持转换kafka中的消息
|
||||
- open-im-server/v3/tools/data-conversion/chat/chat.go
|
||||
- 需要手动创建v3版本数据库,字符集`utf8mb4`
|
||||
- main.go数据库信息
|
||||
```go
|
||||
var (
|
||||
usernameV2 = "root" // v2版本mysql用户名
|
||||
passwordV2 = "openIM" // v2版本mysql密码
|
||||
addrV2 = "127.0.0.1:13306" // v2版本mysql地址
|
||||
databaseV2 = "admin_chat" // v2版本mysql数据库名字
|
||||
)
|
||||
|
||||
var (
|
||||
usernameV3 = "root" // v3版本mysql用户名
|
||||
passwordV3 = "openIM123" // v3版本mysql密码
|
||||
addrV3 = "127.0.0.1:13306" // v3版本mysql地址
|
||||
databaseV3 = "openim_enterprise" // v3版本mysql数据库名字
|
||||
)
|
||||
```
|
||||
```shell
|
||||
go run chat.go
|
||||
```
|
||||
@ -1,88 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package data_conversion
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/IBM/sarama"
|
||||
)
|
||||
|
||||
var (
|
||||
topic = "latestMsgToRedis"
|
||||
addr = "127.0.0.1:9092"
|
||||
)
|
||||
|
||||
var (
|
||||
consumer sarama.Consumer
|
||||
producer sarama.SyncProducer
|
||||
)
|
||||
|
||||
func init() {
|
||||
// Producer
|
||||
config := sarama.NewConfig() // Instantiate a sarama Config
|
||||
config.Producer.Return.Successes = true // Whether to enable the successes channel to be notified after the message is sent successfully
|
||||
config.Producer.Return.Errors = true
|
||||
config.Producer.RequiredAcks = sarama.WaitForAll // Set producer Message Reply level 0 1 all
|
||||
config.Producer.Partitioner = sarama.NewHashPartitioner // Set the hash-key automatic hash partition. When sending a message, you must specify the key value of the message. If there is no key, the partition will be selected randomly
|
||||
|
||||
client, err := sarama.NewSyncProducer([]string{addr}, config)
|
||||
if err != nil {
|
||||
fmt.Println("producer closed, err:", err)
|
||||
}
|
||||
producer = client
|
||||
|
||||
// Consumer
|
||||
consumerT, err := sarama.NewConsumer([]string{addr}, sarama.NewConfig())
|
||||
if err != nil {
|
||||
fmt.Printf("fail to start consumer, err:%v\n", err)
|
||||
}
|
||||
consumer = consumerT
|
||||
}
|
||||
|
||||
func SendMessage() {
|
||||
// construct a message
|
||||
msg := &sarama.ProducerMessage{}
|
||||
msg.Topic = topic
|
||||
msg.Value = sarama.StringEncoder("this is a test log")
|
||||
|
||||
// Send a message
|
||||
pid, offset, err := producer.SendMessage(msg)
|
||||
if err != nil {
|
||||
fmt.Println("send msg failed, err:", err)
|
||||
}
|
||||
fmt.Printf("pid:%v offset:%v\n", pid, offset)
|
||||
}
|
||||
|
||||
func GetMessage() {
|
||||
partitionList, err := consumer.Partitions(topic) // Get all partitions according to topic
|
||||
if err != nil {
|
||||
fmt.Printf("fail to get list of partition:err%v\n", err)
|
||||
}
|
||||
fmt.Println(partitionList)
|
||||
for partition := range partitionList { // iterate over all partitions
|
||||
// Create a corresponding partition consumer for each partition
|
||||
pc, err := consumer.ConsumePartition(topic, int32(partition), sarama.OffsetNewest)
|
||||
if err != nil {
|
||||
fmt.Printf("failed to start consumer for partition %d,err:%v\n", partition, err)
|
||||
}
|
||||
// Asynchronously consume information from each partition
|
||||
go func(sarama.PartitionConsumer) {
|
||||
for msg := range pc.Messages() {
|
||||
fmt.Printf("Partition:%d Offset:%d Key:%v Value:%v", msg.Partition, msg.Offset, msg.Key, msg.Value)
|
||||
}
|
||||
}(pc)
|
||||
}
|
||||
}
|
||||
@ -1,15 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package data_conversion
|
||||
@ -1,228 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package data_conversion
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/OpenIMSDK/tools/log"
|
||||
"gorm.io/driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"github.com/openimsdk/open-im-server/v3/pkg/common/db/table/relation"
|
||||
)
|
||||
|
||||
var (
|
||||
MysqlDb_v2 *gorm.DB
|
||||
MysqlDb_v3 *gorm.DB
|
||||
)
|
||||
|
||||
const (
|
||||
username_v2 = "root"
|
||||
password_v2 = "123456"
|
||||
ip_v2 = "127.0.0.1:3306"
|
||||
database_v2 = "openim_v2"
|
||||
)
|
||||
|
||||
const (
|
||||
username_v3 = "root"
|
||||
password_v3 = "123456"
|
||||
ip_v3 = "127.0.0.1:3306"
|
||||
database_v3 = "openim_v3"
|
||||
)
|
||||
|
||||
func init() {
|
||||
dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local",
|
||||
username_v2,
|
||||
password_v2,
|
||||
ip_v2,
|
||||
database_v2,
|
||||
)
|
||||
db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})
|
||||
MysqlDb_v2 = db
|
||||
if err != nil {
|
||||
log.ZDebug(context.Background(), "err", err)
|
||||
}
|
||||
|
||||
dsn_v3 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local",
|
||||
username_v3,
|
||||
password_v3,
|
||||
ip_v3,
|
||||
database_v3,
|
||||
)
|
||||
db_v3, err := gorm.Open(mysql.Open(dsn_v3), &gorm.Config{})
|
||||
MysqlDb_v3 = db_v3
|
||||
if err != nil {
|
||||
log.ZDebug(context.Background(), "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
func UserConversion() {
|
||||
var count int64
|
||||
var user relation.UserModel
|
||||
MysqlDb_v2.Model(&user).Count(&count)
|
||||
batchSize := 100
|
||||
offset := 0
|
||||
|
||||
for int64(offset) < count {
|
||||
var results []relation.UserModel
|
||||
MysqlDb_v2.Limit(batchSize).Offset(offset).Find(&results)
|
||||
// Process query results
|
||||
fmt.Println("============================batch data===================", offset, batchSize)
|
||||
fmt.Println(results)
|
||||
MysqlDb_v3.Create(results)
|
||||
fmt.Println("======================================================")
|
||||
offset += batchSize
|
||||
}
|
||||
}
|
||||
|
||||
func FriendConversion() {
|
||||
var count int64
|
||||
var friend relation.FriendModel
|
||||
MysqlDb_v2.Model(&friend).Count(&count)
|
||||
batchSize := 100
|
||||
offset := 0
|
||||
|
||||
for int64(offset) < count {
|
||||
var results []relation.FriendModel
|
||||
MysqlDb_v2.Limit(batchSize).Offset(offset).Find(&results)
|
||||
// Process query results
|
||||
fmt.Println("============================batch data===================", offset, batchSize)
|
||||
fmt.Println(results)
|
||||
MysqlDb_v3.Create(results)
|
||||
fmt.Println("======================================================")
|
||||
offset += batchSize
|
||||
}
|
||||
}
|
||||
|
||||
func RequestConversion() {
|
||||
var count int64
|
||||
var friendRequest relation.FriendRequestModel
|
||||
MysqlDb_v2.Model(&friendRequest).Count(&count)
|
||||
batchSize := 100
|
||||
offset := 0
|
||||
|
||||
for int64(offset) < count {
|
||||
var results []relation.FriendRequestModel
|
||||
MysqlDb_v2.Limit(batchSize).Offset(offset).Find(&results)
|
||||
// Process query results
|
||||
fmt.Println("============================batch data===================", offset, batchSize)
|
||||
fmt.Println(results)
|
||||
MysqlDb_v3.Create(results)
|
||||
fmt.Println("======================================================")
|
||||
offset += batchSize
|
||||
}
|
||||
|
||||
var groupRequest relation.GroupRequestModel
|
||||
MysqlDb_v2.Model(&groupRequest).Count(&count)
|
||||
batchSize = 100
|
||||
offset = 0
|
||||
|
||||
for int64(offset) < count {
|
||||
var results []relation.GroupRequestModel
|
||||
MysqlDb_v2.Limit(batchSize).Offset(offset).Find(&results)
|
||||
// Process query results
|
||||
fmt.Println("============================batch data===================", offset, batchSize)
|
||||
fmt.Println(results)
|
||||
MysqlDb_v3.Create(results)
|
||||
fmt.Println("======================================================")
|
||||
offset += batchSize
|
||||
}
|
||||
}
|
||||
|
||||
func GroupConversion() {
|
||||
var count int64
|
||||
var group relation.GroupModel
|
||||
MysqlDb_v2.Model(&group).Count(&count)
|
||||
batchSize := 100
|
||||
offset := 0
|
||||
|
||||
for int64(offset) < count {
|
||||
var results []relation.GroupModel
|
||||
MysqlDb_v2.Limit(batchSize).Offset(offset).Find(&results)
|
||||
for i, val := range results {
|
||||
temp := time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC)
|
||||
if val.NotificationUpdateTime.Equal(temp) {
|
||||
results[i].NotificationUpdateTime = time.Now()
|
||||
// fmt.Println(val.NotificationUpdateTime)
|
||||
}
|
||||
}
|
||||
// Process query results
|
||||
fmt.Println("============================batch data===================", offset, batchSize)
|
||||
fmt.Println(results)
|
||||
MysqlDb_v3.Create(results)
|
||||
fmt.Println("======================================================")
|
||||
offset += batchSize
|
||||
}
|
||||
}
|
||||
|
||||
func GroupMemberConversion() {
|
||||
var count int64
|
||||
var groupMember relation.GroupMemberModel
|
||||
MysqlDb_v2.Model(&groupMember).Count(&count)
|
||||
batchSize := 100
|
||||
offset := 0
|
||||
|
||||
for int64(offset) < count {
|
||||
var results []relation.GroupMemberModel
|
||||
MysqlDb_v2.Limit(batchSize).Offset(offset).Find(&results)
|
||||
// Process query results
|
||||
fmt.Println("============================batch data===================", offset, batchSize)
|
||||
fmt.Println(results)
|
||||
MysqlDb_v3.Create(results)
|
||||
fmt.Println("======================================================")
|
||||
offset += batchSize
|
||||
}
|
||||
}
|
||||
|
||||
func BlacksConversion() {
|
||||
var count int64
|
||||
var black relation.BlackModel
|
||||
MysqlDb_v2.Model(&black).Count(&count)
|
||||
batchSize := 100
|
||||
offset := 0
|
||||
|
||||
for int64(offset) < count {
|
||||
var results []relation.BlackModel
|
||||
MysqlDb_v2.Limit(batchSize).Offset(offset).Find(&results)
|
||||
// Process query results
|
||||
fmt.Println("============================batch data===================", offset, batchSize)
|
||||
fmt.Println(results)
|
||||
MysqlDb_v3.Create(results)
|
||||
fmt.Println("======================================================")
|
||||
offset += batchSize
|
||||
}
|
||||
}
|
||||
|
||||
func ChatLogsConversion() {
|
||||
var count int64
|
||||
var chat relation.ChatLogModel
|
||||
MysqlDb_v2.Model(&chat).Count(&count)
|
||||
batchSize := 100
|
||||
offset := 0
|
||||
|
||||
for int64(offset) < count {
|
||||
var results []relation.ChatLogModel
|
||||
MysqlDb_v2.Limit(batchSize).Offset(offset).Find(&results)
|
||||
// Process query results
|
||||
fmt.Println("============================batch data===================", offset, batchSize)
|
||||
// fmt.Println(results)
|
||||
MysqlDb_v3.Create(results)
|
||||
fmt.Println("======================================================")
|
||||
offset += batchSize
|
||||
}
|
||||
}
|
||||
@ -1,49 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package data_conversion
|
||||
|
||||
import "testing"
|
||||
|
||||
// pass
|
||||
func TestUserConversion(t *testing.T) {
|
||||
UserConversion()
|
||||
}
|
||||
|
||||
// pass
|
||||
func TestFriendConversion(t *testing.T) {
|
||||
FriendConversion()
|
||||
}
|
||||
|
||||
// pass
|
||||
func TestGroupConversion(t *testing.T) {
|
||||
GroupConversion()
|
||||
GroupMemberConversion()
|
||||
}
|
||||
|
||||
// pass
|
||||
func TestBlacksConversion(t *testing.T) {
|
||||
BlacksConversion()
|
||||
}
|
||||
|
||||
// pass
|
||||
func TestRequestConversion(t *testing.T) {
|
||||
RequestConversion()
|
||||
}
|
||||
|
||||
// pass
|
||||
func TestChatLogsConversion(t *testing.T) {
|
||||
// If the printed result is too long, the console will not display it, but it can run normally
|
||||
ChatLogsConversion()
|
||||
}
|
||||
@ -0,0 +1,65 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package common
|
||||
|
||||
// =================================== V2 =====================================
|
||||
// MySQL
|
||||
// V2.
|
||||
const (
|
||||
UsernameV2 = "root"
|
||||
PasswordV2 = "openIM"
|
||||
IpV2 = "121.5.182.23:13306"
|
||||
DatabaseV2 = "openIM_v2"
|
||||
)
|
||||
|
||||
// V2 chat.
|
||||
const (
|
||||
ChatUsernameV2 = "root"
|
||||
ChatPasswordV2 = "openIM"
|
||||
ChatIpV2 = "121.5.182.23:13306"
|
||||
ChatDatabaseV2 = "admin_chat"
|
||||
)
|
||||
|
||||
// Kafka.
|
||||
const (
|
||||
Topic = "ws2ms_chat"
|
||||
KafkaAddr = "121.5.182.23:9092"
|
||||
)
|
||||
|
||||
// =================================== V3 =====================================
|
||||
// V3.
|
||||
const (
|
||||
UsernameV3 = "root"
|
||||
PasswordV3 = "openIM123"
|
||||
IpV3 = "43.134.63.160:13306"
|
||||
DatabaseV3 = "openIM_v3"
|
||||
)
|
||||
|
||||
// V3 chat.
|
||||
const (
|
||||
ChatUsernameV3 = "root"
|
||||
ChatPasswordV3 = "openIM123"
|
||||
ChatIpV3 = "43.134.63.160:13306"
|
||||
ChatDatabaseV3 = "openim_enterprise"
|
||||
)
|
||||
|
||||
// Zookeeper.
|
||||
const (
|
||||
ZkAddr = "43.134.63.160:12181"
|
||||
ZKSchema = "openim"
|
||||
ZKUsername = ""
|
||||
ZKPassword = ""
|
||||
MsgRpcName = "Msg"
|
||||
)
|
||||
@ -0,0 +1,202 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"github.com/IBM/sarama"
|
||||
"github.com/OpenIMSDK/protocol/constant"
|
||||
"github.com/OpenIMSDK/protocol/msg"
|
||||
"github.com/OpenIMSDK/protocol/sdkws"
|
||||
"github.com/OpenIMSDK/tools/mw"
|
||||
"github.com/golang/protobuf/proto"
|
||||
"github.com/openimsdk/open-im-server/v3/pkg/apistruct"
|
||||
pbmsg "github.com/openimsdk/open-im-server/v3/tools/data-conversion/openim/proto/msg"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials/insecure"
|
||||
"log"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
func main() {
|
||||
|
||||
var (
|
||||
topic = "ws2ms_chat" // v2版本配置文件kafka.topic.ws2ms_chat
|
||||
kafkaAddr = "127.0.0.1:9092" // v2版本配置文件kafka.topic.addr
|
||||
rpcAddr = "127.0.0.1:10130" // v3版本配置文件rpcPort.openImMessagePort
|
||||
adminUserID = "openIM123456" // v3版本管理员userID
|
||||
concurrency = 1 // 并发数量
|
||||
)
|
||||
|
||||
getRpcConn := func() (*grpc.ClientConn, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
|
||||
defer cancel()
|
||||
return grpc.DialContext(ctx, rpcAddr, grpc.WithTransportCredentials(insecure.NewCredentials()), mw.GrpcClient())
|
||||
}
|
||||
conn, err := getRpcConn()
|
||||
if err != nil {
|
||||
log.Println("get rpc conn", err)
|
||||
return
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
msgClient := msg.NewMsgClient(conn)
|
||||
|
||||
conf := sarama.NewConfig()
|
||||
conf.Consumer.Offsets.Initial = sarama.OffsetOldest
|
||||
|
||||
consumer, err := sarama.NewConsumer([]string{kafkaAddr}, conf)
|
||||
if err != nil {
|
||||
log.Println("kafka consumer conn", err)
|
||||
return
|
||||
}
|
||||
partitions, err := consumer.Partitions(topic) // Get all partitions according to topic
|
||||
if err != nil {
|
||||
log.Println("kafka partitions", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(partitions) == 0 {
|
||||
log.Println("kafka partitions is empty")
|
||||
return
|
||||
}
|
||||
log.Println("kafka partitions", partitions)
|
||||
|
||||
msgCh := make(chan *pbmsg.MsgDataToMQ, concurrency*2)
|
||||
|
||||
var kfkWg sync.WaitGroup
|
||||
|
||||
distinct := make(map[string]struct{})
|
||||
var lock sync.Mutex
|
||||
|
||||
for _, partition := range partitions {
|
||||
kfkWg.Add(1)
|
||||
go func(partition int32) {
|
||||
defer kfkWg.Done()
|
||||
pc, err := consumer.ConsumePartition(topic, partition, sarama.OffsetOldest)
|
||||
if err != nil {
|
||||
log.Printf("kafka Consume Partition %d failed %s\n", partition, err)
|
||||
return
|
||||
}
|
||||
defer pc.Close()
|
||||
ch := pc.Messages()
|
||||
for {
|
||||
select {
|
||||
case <-time.After(time.Second * 10): // 10s读取不到就关闭
|
||||
return
|
||||
case message, ok := <-ch:
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
msgFromMQV2 := pbmsg.MsgDataToMQ{}
|
||||
err := proto.Unmarshal(message.Value, &msgFromMQV2)
|
||||
if err != nil {
|
||||
log.Printf("kafka msg partition %d offset %d unmarshal failed %s\n", message.Partition, message.Offset, message.Value)
|
||||
continue
|
||||
}
|
||||
if msgFromMQV2.MsgData == nil || msgFromMQV2.OperationID == "" {
|
||||
continue
|
||||
}
|
||||
if msgFromMQV2.MsgData.ContentType < constant.ContentTypeBegin || msgFromMQV2.MsgData.ContentType > constant.AdvancedText {
|
||||
continue
|
||||
}
|
||||
lock.Lock()
|
||||
_, exist := distinct[msgFromMQV2.MsgData.ClientMsgID]
|
||||
if !exist {
|
||||
distinct[msgFromMQV2.MsgData.ClientMsgID] = struct{}{}
|
||||
}
|
||||
lock.Unlock()
|
||||
if exist {
|
||||
continue
|
||||
}
|
||||
msgCh <- &msgFromMQV2
|
||||
}
|
||||
}
|
||||
}(partition)
|
||||
}
|
||||
|
||||
go func() {
|
||||
kfkWg.Wait()
|
||||
close(msgCh)
|
||||
}()
|
||||
|
||||
var msgWg sync.WaitGroup
|
||||
|
||||
var (
|
||||
success int64
|
||||
failed int64
|
||||
)
|
||||
for i := 0; i < concurrency; i++ {
|
||||
msgWg.Add(1)
|
||||
go func() {
|
||||
defer msgWg.Done()
|
||||
for message := range msgCh {
|
||||
HandlerV2Msg(msgClient, adminUserID, message, &success, &failed)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
msgWg.Wait()
|
||||
log.Printf("total %d success %d failed %d\n", success+failed, success, failed)
|
||||
}
|
||||
|
||||
func HandlerV2Msg(msgClient msg.MsgClient, adminUserID string, msgFromMQV2 *pbmsg.MsgDataToMQ, success *int64, failed *int64) {
|
||||
msgData := &sdkws.MsgData{
|
||||
SendID: msgFromMQV2.MsgData.SendID,
|
||||
RecvID: msgFromMQV2.MsgData.RecvID,
|
||||
GroupID: msgFromMQV2.MsgData.GroupID,
|
||||
ClientMsgID: msgFromMQV2.MsgData.ClientMsgID,
|
||||
ServerMsgID: msgFromMQV2.MsgData.ServerMsgID,
|
||||
SenderPlatformID: msgFromMQV2.MsgData.SenderPlatformID,
|
||||
SenderNickname: msgFromMQV2.MsgData.SenderNickname,
|
||||
SenderFaceURL: msgFromMQV2.MsgData.SenderFaceURL,
|
||||
SessionType: msgFromMQV2.MsgData.SessionType,
|
||||
MsgFrom: msgFromMQV2.MsgData.MsgFrom,
|
||||
ContentType: msgFromMQV2.MsgData.ContentType,
|
||||
SendTime: msgFromMQV2.MsgData.SendTime,
|
||||
CreateTime: msgFromMQV2.MsgData.CreateTime,
|
||||
Status: msgFromMQV2.MsgData.Status,
|
||||
IsRead: false,
|
||||
Options: msgFromMQV2.MsgData.Options,
|
||||
AtUserIDList: msgFromMQV2.MsgData.AtUserIDList,
|
||||
AttachedInfo: msgFromMQV2.MsgData.AttachedInfo,
|
||||
Ex: msgFromMQV2.MsgData.Ex,
|
||||
}
|
||||
|
||||
if msgFromMQV2.MsgData.OfflinePushInfo != nil {
|
||||
msgData.OfflinePushInfo = &sdkws.OfflinePushInfo{
|
||||
Title: msgFromMQV2.MsgData.OfflinePushInfo.Title,
|
||||
Desc: msgFromMQV2.MsgData.OfflinePushInfo.Desc,
|
||||
Ex: msgFromMQV2.MsgData.OfflinePushInfo.Ex,
|
||||
IOSPushSound: msgFromMQV2.MsgData.OfflinePushInfo.IOSPushSound,
|
||||
IOSBadgeCount: msgFromMQV2.MsgData.OfflinePushInfo.IOSBadgeCount,
|
||||
SignalInfo: "",
|
||||
}
|
||||
}
|
||||
switch msgData.ContentType {
|
||||
case constant.Text:
|
||||
data, err := json.Marshal(apistruct.TextElem{
|
||||
Content: string(msgFromMQV2.MsgData.Content),
|
||||
})
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
msgData.Content = data
|
||||
default:
|
||||
msgData.Content = msgFromMQV2.MsgData.Content
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
|
||||
defer cancel()
|
||||
ctx = context.WithValue(context.Background(), constant.OperationID, msgFromMQV2.OperationID)
|
||||
ctx = context.WithValue(ctx, constant.OpUserID, adminUserID)
|
||||
|
||||
resp, err := msgClient.SendMsg(ctx, &msg.SendMsgReq{MsgData: msgData})
|
||||
if err != nil {
|
||||
atomic.AddInt64(failed, 1)
|
||||
log.Printf("send msg %+v failed %s\n", msgData, err)
|
||||
return
|
||||
}
|
||||
atomic.AddInt64(success, 1)
|
||||
log.Printf("send msg success %+v resp %+v\n", msgData, resp)
|
||||
}
|
||||
@ -0,0 +1,69 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/openimsdk/open-im-server/v3/tools/data-conversion/openim/mysql/conversion"
|
||||
"github.com/openimsdk/open-im-server/v3/tools/data-conversion/utils"
|
||||
"gorm.io/driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
"log"
|
||||
)
|
||||
|
||||
func main() {
|
||||
var (
|
||||
usernameV2 = "root" // v2版本mysql用户名
|
||||
passwordV2 = "openIM" // v2版本mysql密码
|
||||
addrV2 = "127.0.0.1:13306" // v2版本mysql地址
|
||||
databaseV2 = "openIM_v2" // v2版本mysql数据库名字
|
||||
)
|
||||
|
||||
var (
|
||||
usernameV3 = "root" // v3版本mysql用户名
|
||||
passwordV3 = "openIM123" // v3版本mysql密码
|
||||
addrV3 = "127.0.0.1:13306" // v3版本mysql地址
|
||||
databaseV3 = "openIM_v3" // v3版本mysql数据库名字
|
||||
)
|
||||
|
||||
var concurrency = 1 // 并发数量
|
||||
|
||||
log.SetFlags(log.LstdFlags | log.Llongfile)
|
||||
dsnV2 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", usernameV2, passwordV2, addrV2, databaseV2)
|
||||
dsnV3 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", usernameV3, passwordV3, addrV3, databaseV3)
|
||||
dbV2, err := gorm.Open(mysql.Open(dsnV2), &gorm.Config{Logger: logger.Discard})
|
||||
if err != nil {
|
||||
log.Println("open v2 db failed", err)
|
||||
return
|
||||
}
|
||||
dbV3, err := gorm.Open(mysql.Open(dsnV3), &gorm.Config{Logger: logger.Discard})
|
||||
if err != nil {
|
||||
log.Println("open v3 db failed", err)
|
||||
return
|
||||
}
|
||||
|
||||
var tasks utils.TakeList
|
||||
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Friend) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.FriendRequest) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Group) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.GroupMember) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.GroupRequest) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.User) })
|
||||
|
||||
utils.RunTask(concurrency, tasks)
|
||||
|
||||
}
|
||||
@ -0,0 +1,52 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/openimsdk/open-im-server/v3/tools/data-conversion/openim/mysql/conversion"
|
||||
"github.com/openimsdk/open-im-server/v3/tools/data-conversion/utils"
|
||||
"gorm.io/driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
"log"
|
||||
)
|
||||
|
||||
func Cmd() {
|
||||
var (
|
||||
usernameV2 = "root"
|
||||
passwordV2 = "openIM"
|
||||
addrV2 = "121.5.182.23:13306"
|
||||
databaseV2 = "openIM_v2"
|
||||
)
|
||||
|
||||
var (
|
||||
usernameV3 = "root"
|
||||
passwordV3 = "openIM123"
|
||||
addrV3 = "203.56.175.233:13306"
|
||||
databaseV3 = "openIM_v3"
|
||||
)
|
||||
log.SetFlags(log.LstdFlags | log.Llongfile)
|
||||
dsnV2 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", usernameV2, passwordV2, addrV2, databaseV2)
|
||||
dsnV3 := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8mb4&parseTime=True&loc=Local", usernameV3, passwordV3, addrV3, databaseV3)
|
||||
dbV2, err := gorm.Open(mysql.Open(dsnV2), &gorm.Config{Logger: logger.Discard})
|
||||
if err != nil {
|
||||
log.Println("open v2 db failed", err)
|
||||
return
|
||||
}
|
||||
dbV3, err := gorm.Open(mysql.Open(dsnV3), &gorm.Config{Logger: logger.Discard})
|
||||
if err != nil {
|
||||
log.Println("open v3 db failed", err)
|
||||
return
|
||||
}
|
||||
|
||||
var tasks utils.TakeList
|
||||
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Friend) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.FriendRequest) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.Group) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.GroupMember) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.GroupRequest) })
|
||||
tasks.Append(func() (string, error) { return utils.FindAndInsert(dbV2, dbV3, conversion.User) })
|
||||
|
||||
utils.RunTask(4, tasks)
|
||||
|
||||
}
|
||||
@ -0,0 +1,110 @@
|
||||
package conversion
|
||||
|
||||
import (
|
||||
"github.com/OpenIMSDK/protocol/constant"
|
||||
v3 "github.com/openimsdk/open-im-server/v3/pkg/common/db/table/relation"
|
||||
v2 "github.com/openimsdk/open-im-server/v3/tools/data-conversion/openim/mysql/v2"
|
||||
"github.com/openimsdk/open-im-server/v3/tools/data-conversion/utils"
|
||||
)
|
||||
|
||||
func Friend(v v2.Friend) (v3.FriendModel, bool) {
|
||||
utils.InitTime(&v.CreateTime)
|
||||
return v3.FriendModel{
|
||||
OwnerUserID: v.OwnerUserID,
|
||||
FriendUserID: v.FriendUserID,
|
||||
Remark: v.Remark,
|
||||
CreateTime: v.CreateTime,
|
||||
AddSource: v.AddSource,
|
||||
OperatorUserID: v.OperatorUserID,
|
||||
Ex: v.Ex,
|
||||
}, true
|
||||
}
|
||||
|
||||
func FriendRequest(v v2.FriendRequest) (v3.FriendRequestModel, bool) {
|
||||
utils.InitTime(&v.CreateTime, &v.HandleTime)
|
||||
return v3.FriendRequestModel{
|
||||
FromUserID: v.FromUserID,
|
||||
ToUserID: v.ToUserID,
|
||||
HandleResult: v.HandleResult,
|
||||
ReqMsg: v.ReqMsg,
|
||||
CreateTime: v.CreateTime,
|
||||
HandlerUserID: v.HandlerUserID,
|
||||
HandleMsg: v.HandleMsg,
|
||||
HandleTime: v.HandleTime,
|
||||
Ex: v.Ex,
|
||||
}, true
|
||||
}
|
||||
|
||||
func Group(v v2.Group) (v3.GroupModel, bool) {
|
||||
switch v.GroupType {
|
||||
case constant.WorkingGroup, constant.NormalGroup:
|
||||
v.GroupType = constant.WorkingGroup
|
||||
default:
|
||||
return v3.GroupModel{}, false
|
||||
}
|
||||
utils.InitTime(&v.CreateTime, &v.NotificationUpdateTime)
|
||||
return v3.GroupModel{
|
||||
GroupID: v.GroupID,
|
||||
GroupName: v.GroupName,
|
||||
Notification: v.Notification,
|
||||
Introduction: v.Introduction,
|
||||
FaceURL: v.FaceURL,
|
||||
CreateTime: v.CreateTime,
|
||||
Ex: v.Ex,
|
||||
Status: v.Status,
|
||||
CreatorUserID: v.CreatorUserID,
|
||||
GroupType: v.GroupType,
|
||||
NeedVerification: v.NeedVerification,
|
||||
LookMemberInfo: v.LookMemberInfo,
|
||||
ApplyMemberFriend: v.ApplyMemberFriend,
|
||||
NotificationUpdateTime: v.NotificationUpdateTime,
|
||||
NotificationUserID: v.NotificationUserID,
|
||||
}, true
|
||||
}
|
||||
|
||||
func GroupMember(v v2.GroupMember) (v3.GroupMemberModel, bool) {
|
||||
utils.InitTime(&v.JoinTime, &v.MuteEndTime)
|
||||
return v3.GroupMemberModel{
|
||||
GroupID: v.GroupID,
|
||||
UserID: v.UserID,
|
||||
Nickname: v.Nickname,
|
||||
FaceURL: v.FaceURL,
|
||||
RoleLevel: v.RoleLevel,
|
||||
JoinTime: v.JoinTime,
|
||||
JoinSource: v.JoinSource,
|
||||
InviterUserID: v.InviterUserID,
|
||||
OperatorUserID: v.OperatorUserID,
|
||||
MuteEndTime: v.MuteEndTime,
|
||||
Ex: v.Ex,
|
||||
}, true
|
||||
}
|
||||
|
||||
func GroupRequest(v v2.GroupRequest) (v3.GroupRequestModel, bool) {
|
||||
utils.InitTime(&v.ReqTime, &v.HandledTime)
|
||||
return v3.GroupRequestModel{
|
||||
UserID: v.UserID,
|
||||
GroupID: v.GroupID,
|
||||
HandleResult: v.HandleResult,
|
||||
ReqMsg: v.ReqMsg,
|
||||
HandledMsg: v.HandledMsg,
|
||||
ReqTime: v.ReqTime,
|
||||
HandleUserID: v.HandleUserID,
|
||||
HandledTime: v.HandledTime,
|
||||
JoinSource: v.JoinSource,
|
||||
InviterUserID: v.InviterUserID,
|
||||
Ex: v.Ex,
|
||||
}, true
|
||||
}
|
||||
|
||||
func User(v v2.User) (v3.UserModel, bool) {
|
||||
utils.InitTime(&v.CreateTime)
|
||||
return v3.UserModel{
|
||||
UserID: v.UserID,
|
||||
Nickname: v.Nickname,
|
||||
FaceURL: v.FaceURL,
|
||||
Ex: v.Ex,
|
||||
CreateTime: v.CreateTime,
|
||||
AppMangerLevel: v.AppMangerLevel,
|
||||
GlobalRecvMsgOpt: v.GlobalRecvMsgOpt,
|
||||
}, true
|
||||
}
|
||||
@ -0,0 +1,91 @@
|
||||
package db
|
||||
|
||||
import "time"
|
||||
|
||||
type Friend struct {
|
||||
OwnerUserID string `gorm:"column:owner_user_id;primary_key;size:64"`
|
||||
FriendUserID string `gorm:"column:friend_user_id;primary_key;size:64"`
|
||||
Remark string `gorm:"column:remark;size:255"`
|
||||
CreateTime time.Time `gorm:"column:create_time"`
|
||||
AddSource int32 `gorm:"column:add_source"`
|
||||
OperatorUserID string `gorm:"column:operator_user_id;size:64"`
|
||||
Ex string `gorm:"column:ex;size:1024"`
|
||||
}
|
||||
|
||||
type FriendRequest struct {
|
||||
FromUserID string `gorm:"column:from_user_id;primary_key;size:64"`
|
||||
ToUserID string `gorm:"column:to_user_id;primary_key;size:64"`
|
||||
HandleResult int32 `gorm:"column:handle_result"`
|
||||
ReqMsg string `gorm:"column:req_msg;size:255"`
|
||||
CreateTime time.Time `gorm:"column:create_time"`
|
||||
HandlerUserID string `gorm:"column:handler_user_id;size:64"`
|
||||
HandleMsg string `gorm:"column:handle_msg;size:255"`
|
||||
HandleTime time.Time `gorm:"column:handle_time"`
|
||||
Ex string `gorm:"column:ex;size:1024"`
|
||||
}
|
||||
|
||||
func (FriendRequest) TableName() string {
|
||||
return "friend_requests"
|
||||
}
|
||||
|
||||
type Group struct {
|
||||
GroupID string `gorm:"column:group_id;primary_key;size:64" json:"groupID" binding:"required"`
|
||||
GroupName string `gorm:"column:name;size:255" json:"groupName"`
|
||||
Notification string `gorm:"column:notification;size:255" json:"notification"`
|
||||
Introduction string `gorm:"column:introduction;size:255" json:"introduction"`
|
||||
FaceURL string `gorm:"column:face_url;size:255" json:"faceURL"`
|
||||
CreateTime time.Time `gorm:"column:create_time;index:create_time"`
|
||||
Ex string `gorm:"column:ex" json:"ex;size:1024" json:"ex"`
|
||||
Status int32 `gorm:"column:status"`
|
||||
CreatorUserID string `gorm:"column:creator_user_id;size:64"`
|
||||
GroupType int32 `gorm:"column:group_type"`
|
||||
NeedVerification int32 `gorm:"column:need_verification"`
|
||||
LookMemberInfo int32 `gorm:"column:look_member_info" json:"lookMemberInfo"`
|
||||
ApplyMemberFriend int32 `gorm:"column:apply_member_friend" json:"applyMemberFriend"`
|
||||
NotificationUpdateTime time.Time `gorm:"column:notification_update_time"`
|
||||
NotificationUserID string `gorm:"column:notification_user_id;size:64"`
|
||||
}
|
||||
|
||||
type GroupMember struct {
|
||||
GroupID string `gorm:"column:group_id;primary_key;size:64"`
|
||||
UserID string `gorm:"column:user_id;primary_key;size:64"`
|
||||
Nickname string `gorm:"column:nickname;size:255"`
|
||||
FaceURL string `gorm:"column:user_group_face_url;size:255"`
|
||||
RoleLevel int32 `gorm:"column:role_level"`
|
||||
JoinTime time.Time `gorm:"column:join_time"`
|
||||
JoinSource int32 `gorm:"column:join_source"`
|
||||
InviterUserID string `gorm:"column:inviter_user_id;size:64"`
|
||||
OperatorUserID string `gorm:"column:operator_user_id;size:64"`
|
||||
MuteEndTime time.Time `gorm:"column:mute_end_time"`
|
||||
Ex string `gorm:"column:ex;size:1024"`
|
||||
}
|
||||
|
||||
type GroupRequest struct {
|
||||
UserID string `gorm:"column:user_id;primary_key;size:64"`
|
||||
GroupID string `gorm:"column:group_id;primary_key;size:64"`
|
||||
HandleResult int32 `gorm:"column:handle_result"`
|
||||
ReqMsg string `gorm:"column:req_msg;size:1024"`
|
||||
HandledMsg string `gorm:"column:handle_msg;size:1024"`
|
||||
ReqTime time.Time `gorm:"column:req_time"`
|
||||
HandleUserID string `gorm:"column:handle_user_id;size:64"`
|
||||
HandledTime time.Time `gorm:"column:handle_time"`
|
||||
JoinSource int32 `gorm:"column:join_source"`
|
||||
InviterUserID string `gorm:"column:inviter_user_id;size:64"`
|
||||
Ex string `gorm:"column:ex;size:1024"`
|
||||
}
|
||||
|
||||
type User struct {
|
||||
UserID string `gorm:"column:user_id;primary_key;size:64"`
|
||||
Nickname string `gorm:"column:name;size:255"`
|
||||
FaceURL string `gorm:"column:face_url;size:255"`
|
||||
Gender int32 `gorm:"column:gender"`
|
||||
PhoneNumber string `gorm:"column:phone_number;size:32"`
|
||||
Birth time.Time `gorm:"column:birth"`
|
||||
Email string `gorm:"column:email;size:64"`
|
||||
Ex string `gorm:"column:ex;size:1024"`
|
||||
CreateTime time.Time `gorm:"column:create_time;index:create_time"`
|
||||
AppMangerLevel int32 `gorm:"column:app_manger_level"`
|
||||
GlobalRecvMsgOpt int32 `gorm:"column:global_recv_msg_opt"`
|
||||
|
||||
status int32 `gorm:"column:status"`
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,315 @@
|
||||
syntax = "proto3";
|
||||
import "Open-IM-Server/pkg/proto/sdk_ws/ws.proto";
|
||||
import "Open-IM-Server/pkg/proto/sdk_ws/wrappers.proto";
|
||||
option go_package = "Open_IM/pkg/proto/msg;msg";
|
||||
package msg;
|
||||
|
||||
message MsgDataToMQ{
|
||||
string token =1;
|
||||
string operationID = 2;
|
||||
server_api_params.MsgData msgData = 3;
|
||||
}
|
||||
|
||||
|
||||
message MsgDataToDB {
|
||||
server_api_params.MsgData msgData = 1;
|
||||
string operationID = 2;
|
||||
|
||||
}
|
||||
message PushMsgDataToMQ{
|
||||
string OperationID = 1;
|
||||
server_api_params.MsgData msgData = 2;
|
||||
string pushToUserID = 3;
|
||||
}
|
||||
message MsgDataToMongoByMQ{
|
||||
uint64 lastSeq =1;
|
||||
string aggregationID = 2;
|
||||
repeated MsgDataToMQ messageList = 3;
|
||||
string triggerID = 4;
|
||||
|
||||
|
||||
}
|
||||
|
||||
//message PullMessageReq {
|
||||
// string UserID = 1;
|
||||
// int64 SeqBegin = 2;
|
||||
// int64 SeqEnd = 3;
|
||||
// string OperationID = 4;
|
||||
//}
|
||||
//
|
||||
//message PullMessageResp {
|
||||
// int32 ErrCode = 1;
|
||||
// string ErrMsg = 2;
|
||||
// int64 MaxSeq = 3;
|
||||
// int64 MinSeq = 4;
|
||||
// repeated GatherFormat SingleUserMsg = 5;
|
||||
// repeated GatherFormat GroupUserMsg = 6;
|
||||
//}
|
||||
//message PullMessageBySeqListReq{
|
||||
// string UserID = 1;
|
||||
// string OperationID = 2;
|
||||
// repeated int64 seqList =3;
|
||||
//}
|
||||
message GetMaxAndMinSeqReq {
|
||||
string UserID = 1;
|
||||
string OperationID = 2;
|
||||
}
|
||||
message GetMaxAndMinSeqResp {
|
||||
int32 ErrCode = 1;
|
||||
string ErrMsg = 2;
|
||||
uint32 MaxSeq = 3;
|
||||
uint32 MinSeq = 4;
|
||||
}
|
||||
|
||||
message SendMsgReq {
|
||||
|
||||
string token =1;
|
||||
string operationID = 2;
|
||||
server_api_params.MsgData msgData = 3;
|
||||
|
||||
|
||||
}
|
||||
|
||||
message SendMsgResp {
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
string serverMsgID = 4;
|
||||
string clientMsgID = 5;
|
||||
int64 sendTime = 6;
|
||||
string ex = 7;
|
||||
}
|
||||
|
||||
|
||||
message ClearMsgReq{
|
||||
string userID = 1;
|
||||
string opUserID = 2;
|
||||
string operationID = 3;
|
||||
}
|
||||
|
||||
|
||||
message ClearMsgResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
}
|
||||
|
||||
message SetMsgMinSeqReq{
|
||||
string userID = 1;
|
||||
string groupID = 2;
|
||||
uint32 minSeq = 3;
|
||||
string operationID = 4;
|
||||
string opUserID = 5;
|
||||
}
|
||||
message SetMsgMinSeqResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
}
|
||||
|
||||
message SetSendMsgStatusReq{
|
||||
string operationID = 1;
|
||||
int32 status = 2;
|
||||
}
|
||||
|
||||
message SetSendMsgStatusResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
}
|
||||
|
||||
message GetSendMsgStatusReq{
|
||||
string operationID = 1;
|
||||
}
|
||||
|
||||
message GetSendMsgStatusResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
int32 status = 3;
|
||||
}
|
||||
message DelSuperGroupMsgReq{
|
||||
string opUserID = 1;
|
||||
string userID = 2;
|
||||
string groupID = 3;
|
||||
string operationID = 4;
|
||||
}
|
||||
message DelSuperGroupMsgResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
}
|
||||
message GetSuperGroupMsgReq{
|
||||
string operationID = 1;
|
||||
uint32 Seq = 2;
|
||||
string groupID = 3;
|
||||
|
||||
}
|
||||
message GetSuperGroupMsgResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
server_api_params.MsgData msgData = 3;
|
||||
}
|
||||
message GetWriteDiffMsgReq{
|
||||
string operationID = 1;
|
||||
uint32 Seq = 2;
|
||||
|
||||
}
|
||||
message GetWriteDiffMsgResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
server_api_params.MsgData msgData = 3;
|
||||
}
|
||||
|
||||
message ModifyMessageReactionExtensionsReq {
|
||||
string operationID = 1;
|
||||
string sourceID = 2;
|
||||
string opUserID = 3;
|
||||
int32 sessionType = 4;
|
||||
map <string, server_api_params.KeyValue>reactionExtensionList = 5;
|
||||
string clientMsgID = 6;
|
||||
google.protobuf.StringValue ex = 7;
|
||||
google.protobuf.StringValue attachedInfo = 8;
|
||||
bool isReact = 9;
|
||||
bool isExternalExtensions = 10;
|
||||
int64 msgFirstModifyTime = 11;
|
||||
}
|
||||
message SetMessageReactionExtensionsReq {
|
||||
string operationID = 1;
|
||||
string sourceID = 2;
|
||||
string opUserID = 3;
|
||||
int32 opUserIDPlatformID = 4;
|
||||
int32 sessionType = 5;
|
||||
map <string, server_api_params.KeyValue>reactionExtensionList = 6;
|
||||
string clientMsgID = 7;
|
||||
google.protobuf.StringValue ex = 8;
|
||||
google.protobuf.StringValue attachedInfo = 9;
|
||||
bool isReact = 10;
|
||||
bool isExternalExtensions = 11;
|
||||
int64 msgFirstModifyTime = 12;
|
||||
}
|
||||
message SetMessageReactionExtensionsResp {
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
string clientMsgID = 3;
|
||||
int64 msgFirstModifyTime = 4;
|
||||
bool isReact = 5;
|
||||
repeated KeyValueResp result = 6;
|
||||
}
|
||||
message AddMessageReactionExtensionsReq {
|
||||
string operationID = 1;
|
||||
string sourceID = 2;
|
||||
string opUserID = 3;
|
||||
int32 opUserIDPlatformID = 4;
|
||||
int32 sessionType = 5;
|
||||
map <string, server_api_params.KeyValue>reactionExtensionList = 6;
|
||||
string clientMsgID = 7;
|
||||
google.protobuf.StringValue ex = 8;
|
||||
google.protobuf.StringValue attachedInfo = 9;
|
||||
bool isReact = 10;
|
||||
bool isExternalExtensions = 11;
|
||||
int64 msgFirstModifyTime = 12;
|
||||
uint32 seq = 13;
|
||||
}
|
||||
message AddMessageReactionExtensionsResp {
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
string clientMsgID = 3;
|
||||
int64 msgFirstModifyTime = 4;
|
||||
bool isReact = 5;
|
||||
repeated KeyValueResp result = 6;
|
||||
}
|
||||
|
||||
|
||||
message GetMessageListReactionExtensionsReq {
|
||||
string operationID = 1;
|
||||
string opUserID = 2;
|
||||
string sourceID = 3;
|
||||
int32 sessionType = 4;
|
||||
bool isExternalExtensions = 5;
|
||||
message MessageReactionKey {
|
||||
string clientMsgID = 1;
|
||||
int64 msgFirstModifyTime = 2;
|
||||
}
|
||||
repeated string typeKeyList = 6;
|
||||
repeated MessageReactionKey messageReactionKeyList = 7;
|
||||
}
|
||||
message GetMessageListReactionExtensionsResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
repeated SingleMessageExtensionResult singleMessageResult =3;
|
||||
|
||||
}
|
||||
message SingleMessageExtensionResult {
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
map <string, server_api_params.KeyValue>reactionExtensionList = 3;
|
||||
string clientMsgID = 4;
|
||||
}
|
||||
|
||||
|
||||
message ModifyMessageReactionExtensionsResp {
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
repeated ExtendMsgResp successList = 3;
|
||||
repeated ExtendMsgResp failedList = 4;
|
||||
}
|
||||
|
||||
message DeleteMessageListReactionExtensionsReq {
|
||||
string operationID = 1;
|
||||
string opUserID = 2;
|
||||
string sourceID = 3;
|
||||
int32 opUserIDPlatformID = 4;
|
||||
int32 sessionType = 5;
|
||||
string clientMsgID = 6;
|
||||
bool isExternalExtensions = 7;
|
||||
int64 msgFirstModifyTime = 8;
|
||||
repeated server_api_params.KeyValue reactionExtensionList = 9;
|
||||
}
|
||||
|
||||
message DeleteMessageListReactionExtensionsResp {
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
repeated KeyValueResp result = 6;
|
||||
}
|
||||
|
||||
message ExtendMsgResp {
|
||||
ExtendMsg extendMsg = 1;
|
||||
int32 errCode = 2;
|
||||
string errMsg = 3;
|
||||
}
|
||||
|
||||
message ExtendMsg {
|
||||
map <string, KeyValueResp>reactionExtensionList = 1;
|
||||
string clientMsgID = 2;
|
||||
int64 msgFirstModifyTime = 3;
|
||||
string attachedInfo = 4;
|
||||
string ex = 5;
|
||||
}
|
||||
|
||||
message KeyValueResp {
|
||||
server_api_params.KeyValue keyValue = 1;
|
||||
int32 errCode = 2;
|
||||
string errMsg = 3;
|
||||
}
|
||||
|
||||
message MsgDataToModifyByMQ{
|
||||
string aggregationID = 1;
|
||||
repeated MsgDataToMQ messageList = 2;
|
||||
string triggerID = 3;
|
||||
}
|
||||
|
||||
|
||||
service msg {
|
||||
rpc GetMaxAndMinSeq(server_api_params.GetMaxAndMinSeqReq) returns(server_api_params.GetMaxAndMinSeqResp);
|
||||
rpc PullMessageBySeqList(server_api_params.PullMessageBySeqListReq) returns(server_api_params.PullMessageBySeqListResp);
|
||||
rpc SendMsg(SendMsgReq) returns(SendMsgResp);
|
||||
rpc DelMsgList(server_api_params.DelMsgListReq) returns(server_api_params.DelMsgListResp);
|
||||
rpc DelSuperGroupMsg(DelSuperGroupMsgReq) returns(DelSuperGroupMsgResp);
|
||||
rpc ClearMsg(ClearMsgReq) returns(ClearMsgResp);
|
||||
rpc SetMsgMinSeq(SetMsgMinSeqReq) returns(SetMsgMinSeqResp);
|
||||
rpc SetSendMsgStatus(SetSendMsgStatusReq) returns(SetSendMsgStatusResp);
|
||||
rpc GetSendMsgStatus(GetSendMsgStatusReq) returns(GetSendMsgStatusResp);
|
||||
rpc GetSuperGroupMsg(GetSuperGroupMsgReq) returns(GetSuperGroupMsgResp);
|
||||
rpc GetWriteDiffMsg(GetWriteDiffMsgReq) returns(GetWriteDiffMsgResp);
|
||||
|
||||
// modify msg
|
||||
rpc SetMessageReactionExtensions(SetMessageReactionExtensionsReq) returns(SetMessageReactionExtensionsResp);
|
||||
rpc GetMessageListReactionExtensions(GetMessageListReactionExtensionsReq) returns(GetMessageListReactionExtensionsResp);
|
||||
rpc AddMessageReactionExtensions(AddMessageReactionExtensionsReq) returns(AddMessageReactionExtensionsResp);
|
||||
rpc DeleteMessageReactionExtensions(DeleteMessageListReactionExtensionsReq) returns(DeleteMessageListReactionExtensionsResp);
|
||||
}
|
||||
@ -0,0 +1,123 @@
|
||||
// Protocol Buffers - Google's data interchange format
|
||||
// Copyright 2008 Google Inc. All rights reserved.
|
||||
// https://developers.google.com/protocol-buffers/
|
||||
//
|
||||
// Redistribution and use in source and binary forms, with or without
|
||||
// modification, are permitted provided that the following conditions are
|
||||
// met:
|
||||
//
|
||||
// * Redistributions of source code must retain the above copyright
|
||||
// notice, this list of conditions and the following disclaimer.
|
||||
// * Redistributions in binary form must reproduce the above
|
||||
// copyright notice, this list of conditions and the following disclaimer
|
||||
// in the documentation and/or other materials provided with the
|
||||
// distribution.
|
||||
// * Neither the name of Google Inc. nor the names of its
|
||||
// contributors may be used to endorse or promote products derived from
|
||||
// this software without specific prior written permission.
|
||||
//
|
||||
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
// Wrappers for primitive (non-message) types. These types are useful
|
||||
// for embedding primitives in the `google.protobuf.Any` type and for places
|
||||
// where we need to distinguish between the absence of a primitive
|
||||
// typed field and its default value.
|
||||
//
|
||||
// These wrappers have no meaningful use within repeated fields as they lack
|
||||
// the ability to detect presence on individual elements.
|
||||
// These wrappers have no meaningful use within a map or a oneof since
|
||||
// individual entries of a map or fields of a oneof can already detect presence.
|
||||
|
||||
syntax = "proto3";
|
||||
|
||||
package google.protobuf;
|
||||
|
||||
option csharp_namespace = "Google.Protobuf.WellKnownTypes";
|
||||
option cc_enable_arenas = true;
|
||||
option go_package = "google.golang.org/protobuf/types/known/wrapperspb";
|
||||
option java_package = "com.google.protobuf";
|
||||
option java_outer_classname = "WrappersProto";
|
||||
option java_multiple_files = true;
|
||||
option objc_class_prefix = "GPB";
|
||||
|
||||
// Wrapper message for `double`.
|
||||
//
|
||||
// The JSON representation for `DoubleValue` is JSON number.
|
||||
message DoubleValue {
|
||||
// The double value.
|
||||
double value = 1;
|
||||
}
|
||||
|
||||
// Wrapper message for `float`.
|
||||
//
|
||||
// The JSON representation for `FloatValue` is JSON number.
|
||||
message FloatValue {
|
||||
// The float value.
|
||||
float value = 1;
|
||||
}
|
||||
|
||||
// Wrapper message for `int64`.
|
||||
//
|
||||
// The JSON representation for `Int64Value` is JSON string.
|
||||
message Int64Value {
|
||||
// The int64 value.
|
||||
int64 value = 1;
|
||||
}
|
||||
|
||||
// Wrapper message for `uint64`.
|
||||
//
|
||||
// The JSON representation for `UInt64Value` is JSON string.
|
||||
message UInt64Value {
|
||||
// The uint64 value.
|
||||
uint64 value = 1;
|
||||
}
|
||||
|
||||
// Wrapper message for `int32`.
|
||||
//
|
||||
// The JSON representation for `Int32Value` is JSON number.
|
||||
message Int32Value {
|
||||
// The int32 value.
|
||||
int32 value = 1;
|
||||
}
|
||||
|
||||
// Wrapper message for `uint32`.
|
||||
//
|
||||
// The JSON representation for `UInt32Value` is JSON number.
|
||||
message UInt32Value {
|
||||
// The uint32 value.
|
||||
uint32 value = 1;
|
||||
}
|
||||
|
||||
// Wrapper message for `bool`.
|
||||
//
|
||||
// The JSON representation for `BoolValue` is JSON `true` and `false`.
|
||||
message BoolValue {
|
||||
// The bool value.
|
||||
bool value = 1;
|
||||
}
|
||||
|
||||
// Wrapper message for `string`.
|
||||
//
|
||||
// The JSON representation for `StringValue` is JSON string.
|
||||
message StringValue {
|
||||
// The string value.
|
||||
string value = 1;
|
||||
}
|
||||
|
||||
// Wrapper message for `bytes`.
|
||||
//
|
||||
// The JSON representation for `BytesValue` is JSON string.
|
||||
message BytesValue {
|
||||
// The bytes value.
|
||||
bytes value = 1;
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,740 @@
|
||||
syntax = "proto3";
|
||||
import "Open-IM-Server/pkg/proto/sdk_ws/wrappers.proto";
|
||||
option go_package = "Open_IM/pkg/proto/sdk_ws;server_api_params";
|
||||
package server_api_params;
|
||||
|
||||
|
||||
////////////////////////////////base///////////////////////////////
|
||||
|
||||
|
||||
message GroupInfo{
|
||||
string groupID = 1;
|
||||
string groupName = 2;
|
||||
string notification = 3;
|
||||
string introduction = 4;
|
||||
string faceURL = 5;
|
||||
string ownerUserID = 6;
|
||||
uint32 createTime = 7;
|
||||
uint32 memberCount = 8;
|
||||
string ex = 9;
|
||||
int32 status = 10;
|
||||
string creatorUserID = 11;
|
||||
int32 groupType = 12;
|
||||
int32 needVerification = 13;
|
||||
int32 lookMemberInfo = 14;
|
||||
int32 applyMemberFriend = 15;
|
||||
uint32 notificationUpdateTime = 16;
|
||||
string notificationUserID = 17;
|
||||
}
|
||||
|
||||
message GroupInfoForSet{
|
||||
string groupID = 1;
|
||||
string groupName = 2;
|
||||
string notification = 3;
|
||||
string introduction = 4;
|
||||
string faceURL = 5;
|
||||
string ex = 6;
|
||||
google.protobuf.Int32Value needVerification = 7;
|
||||
google.protobuf.Int32Value lookMemberInfo = 8;
|
||||
google.protobuf.Int32Value applyMemberFriend = 9;
|
||||
}
|
||||
|
||||
|
||||
message GroupMemberFullInfo {
|
||||
string groupID = 1 ;
|
||||
string userID = 2 ;
|
||||
int32 roleLevel = 3;
|
||||
int32 joinTime = 4;
|
||||
string nickname = 5;
|
||||
string faceURL = 6;
|
||||
int32 appMangerLevel = 7; //if >0
|
||||
int32 joinSource = 8;
|
||||
string operatorUserID = 9;
|
||||
string ex = 10;
|
||||
uint32 muteEndTime = 11;
|
||||
string inviterUserID = 12;
|
||||
}
|
||||
|
||||
message PublicUserInfo{
|
||||
string userID = 1;
|
||||
string nickname = 2;
|
||||
string faceURL = 3;
|
||||
int32 gender = 4;
|
||||
string ex = 5;
|
||||
}
|
||||
|
||||
message UserInfo{
|
||||
string userID = 1;
|
||||
string nickname = 2;
|
||||
string faceURL = 3;
|
||||
int32 gender = 4;
|
||||
string phoneNumber = 5;
|
||||
uint32 birth = 6;
|
||||
string email = 7;
|
||||
string ex = 8;
|
||||
uint32 createTime = 9;
|
||||
int32 appMangerLevel = 10;
|
||||
int32 globalRecvMsgOpt = 11;
|
||||
string birthStr = 12;
|
||||
}
|
||||
|
||||
message FriendInfo{
|
||||
string ownerUserID = 1;
|
||||
string remark = 2;
|
||||
uint32 createTime = 3;
|
||||
UserInfo friendUser = 4;
|
||||
int32 addSource = 5;
|
||||
string operatorUserID = 6;
|
||||
string ex = 7;
|
||||
}
|
||||
|
||||
message BlackInfo{
|
||||
string ownerUserID = 1;
|
||||
uint32 createTime = 2;
|
||||
PublicUserInfo blackUserInfo = 3;
|
||||
int32 addSource = 4;
|
||||
string operatorUserID = 5;
|
||||
string ex = 6;
|
||||
}
|
||||
|
||||
message GroupRequest{
|
||||
PublicUserInfo userInfo = 1;
|
||||
GroupInfo groupInfo = 2;
|
||||
int32 handleResult = 3;
|
||||
string reqMsg = 4;
|
||||
string handleMsg = 5;
|
||||
uint32 reqTime = 6;
|
||||
string handleUserID = 7;
|
||||
uint32 handleTime = 8;
|
||||
string ex = 9;
|
||||
int32 joinSource = 10;
|
||||
string inviterUserID = 11;
|
||||
}
|
||||
|
||||
message FriendRequest{
|
||||
string fromUserID = 1;
|
||||
string fromNickname = 2;
|
||||
string fromFaceURL = 3;
|
||||
int32 fromGender = 4;
|
||||
string toUserID = 5;
|
||||
string toNickname = 6;
|
||||
string toFaceURL = 7;
|
||||
int32 toGender = 8;
|
||||
int32 handleResult = 9;
|
||||
string reqMsg = 10;
|
||||
uint32 createTime = 11;
|
||||
string handlerUserID = 12;
|
||||
string handleMsg = 13;
|
||||
uint32 handleTime = 14;
|
||||
string ex = 15;
|
||||
}
|
||||
|
||||
///////////////////////////////////organization/////////////////////////////////////
|
||||
|
||||
message Department {
|
||||
string departmentID = 1;
|
||||
string faceURL = 2;
|
||||
string name = 3;
|
||||
string parentID = 4;
|
||||
int32 order = 5;
|
||||
int32 departmentType = 6;
|
||||
uint32 createTime = 7;
|
||||
uint32 subDepartmentNum = 8;
|
||||
uint32 memberNum = 9;
|
||||
string ex = 10;
|
||||
}
|
||||
|
||||
|
||||
|
||||
message OrganizationUser {
|
||||
string userID = 1;
|
||||
string nickname = 2;
|
||||
string englishName = 3;
|
||||
string faceURL = 4;
|
||||
int32 gender = 5;
|
||||
string mobile = 6;
|
||||
string telephone = 7;
|
||||
uint32 birth = 8;
|
||||
string email = 9;
|
||||
uint32 createTime = 10;
|
||||
string ex = 11;
|
||||
string birthStr = 12;
|
||||
}
|
||||
|
||||
message DepartmentMember {
|
||||
string userID = 1;
|
||||
string departmentID = 2;
|
||||
int32 order = 3;
|
||||
string position = 4;
|
||||
int32 leader = 5;
|
||||
int32 status = 6;
|
||||
string ex = 7;
|
||||
}
|
||||
|
||||
|
||||
message UserDepartmentMember {
|
||||
OrganizationUser organizationUser = 1;
|
||||
DepartmentMember departmentMember = 2;
|
||||
}
|
||||
|
||||
|
||||
message UserInDepartment {
|
||||
OrganizationUser organizationUser = 1;
|
||||
repeated DepartmentMember departmentMemberList = 2;
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
///////////////////////////////////organization end//////////////////////////////////
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
///////////////////////////////////base end/////////////////////////////////////
|
||||
message PullMessageBySeqListReq{
|
||||
string userID = 1;
|
||||
string operationID = 2;
|
||||
repeated uint32 seqList = 3;
|
||||
map <string, seqList>groupSeqList = 4;
|
||||
}
|
||||
|
||||
message seqList {
|
||||
repeated uint32 seqList = 1;
|
||||
}
|
||||
|
||||
|
||||
message MsgDataList {
|
||||
repeated MsgData msgDataList = 1;
|
||||
}
|
||||
|
||||
message PullMessageBySeqListResp {
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
repeated MsgData list = 3;
|
||||
map<string, MsgDataList> groupMsgDataList = 4;
|
||||
}
|
||||
|
||||
|
||||
|
||||
message GetMaxAndMinSeqReq {
|
||||
repeated string groupIDList = 1;
|
||||
string userID = 2;
|
||||
string operationID = 3;
|
||||
}
|
||||
message MaxAndMinSeq{
|
||||
uint32 maxSeq = 1;
|
||||
uint32 minSeq = 2;
|
||||
}
|
||||
message GetMaxAndMinSeqResp {
|
||||
uint32 maxSeq = 1;
|
||||
uint32 minSeq = 2;
|
||||
int32 errCode = 3;
|
||||
string errMsg = 4;
|
||||
map<string, MaxAndMinSeq> groupMaxAndMinSeq = 5;
|
||||
}
|
||||
|
||||
message UserSendMsgResp {
|
||||
string serverMsgID = 1;
|
||||
string clientMsgID = 2;
|
||||
int64 sendTime = 3;
|
||||
string ex = 4;
|
||||
}
|
||||
|
||||
message MsgData {
|
||||
string sendID = 1;
|
||||
string recvID = 2;
|
||||
string groupID = 3;
|
||||
string clientMsgID = 4;
|
||||
string serverMsgID = 5;
|
||||
int32 senderPlatformID = 6;
|
||||
string senderNickname = 7;
|
||||
string senderFaceURL = 8;
|
||||
int32 sessionType = 9;
|
||||
int32 msgFrom = 10;
|
||||
int32 contentType = 11;
|
||||
bytes content = 12;
|
||||
uint32 seq = 14;
|
||||
int64 sendTime = 15;
|
||||
int64 createTime = 16;
|
||||
int32 status = 17;
|
||||
map<string, bool> options = 18;
|
||||
OfflinePushInfo offlinePushInfo = 19;
|
||||
repeated string atUserIDList = 20;
|
||||
bytes msgDataList = 21;
|
||||
string attachedInfo = 22;
|
||||
string ex = 23;
|
||||
|
||||
bool isReact = 40;
|
||||
bool isExternalExtensions = 41;
|
||||
int64 msgFirstModifyTime = 42;
|
||||
|
||||
}
|
||||
message OfflinePushInfo{
|
||||
string title = 1;
|
||||
string desc = 2;
|
||||
string ex = 3;
|
||||
string iOSPushSound = 4;
|
||||
bool iOSBadgeCount = 5;
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
message TipsComm{
|
||||
bytes detail = 1;
|
||||
string defaultTips = 2;
|
||||
string jsonDetail = 3;
|
||||
}
|
||||
|
||||
//////////////////////group/////////////////////
|
||||
|
||||
|
||||
// OnGroupCreated()
|
||||
message GroupCreatedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
repeated GroupMemberFullInfo memberList = 3;
|
||||
int64 operationTime = 4;
|
||||
GroupMemberFullInfo groupOwnerUser = 5;
|
||||
}
|
||||
|
||||
// OnGroupInfoSet()
|
||||
message GroupInfoSetTips{
|
||||
GroupMemberFullInfo opUser = 1; //who do this
|
||||
int64 muteTime = 2;
|
||||
GroupInfo group = 3;
|
||||
}
|
||||
|
||||
// OnJoinGroupApplication()
|
||||
message JoinGroupApplicationTips{
|
||||
GroupInfo group = 1;
|
||||
PublicUserInfo applicant = 2;
|
||||
string reqMsg = 3;
|
||||
}
|
||||
|
||||
// OnQuitGroup()
|
||||
//Actively leave the group
|
||||
message MemberQuitTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo quitUser = 2;
|
||||
int64 operationTime = 3;
|
||||
}
|
||||
|
||||
|
||||
// OnApplicationGroupAccepted()
|
||||
message GroupApplicationAcceptedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
string handleMsg = 4;
|
||||
int32 receiverAs = 5; // admin(==1) or applicant(==0)
|
||||
}
|
||||
|
||||
// OnApplicationGroupRejected()
|
||||
message GroupApplicationRejectedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
string handleMsg = 4;
|
||||
int32 receiverAs = 5; // admin(==1) or applicant(==0)
|
||||
}
|
||||
|
||||
// OnTransferGroupOwner()
|
||||
message GroupOwnerTransferredTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
GroupMemberFullInfo newGroupOwner = 3;
|
||||
int64 operationTime = 4;
|
||||
}
|
||||
|
||||
|
||||
// OnMemberKicked()
|
||||
message MemberKickedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
repeated GroupMemberFullInfo kickedUserList = 3;
|
||||
int64 operationTime = 4;
|
||||
}
|
||||
|
||||
// OnMemberInvited()
|
||||
message MemberInvitedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
repeated GroupMemberFullInfo invitedUserList = 3;
|
||||
int64 operationTime = 4;
|
||||
}
|
||||
|
||||
//Actively join the group
|
||||
message MemberEnterTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo entrantUser = 2;
|
||||
int64 operationTime = 3;
|
||||
}
|
||||
|
||||
message GroupDismissedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
int64 operationTime = 3;
|
||||
}
|
||||
|
||||
message GroupMemberMutedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
int64 operationTime = 3;
|
||||
GroupMemberFullInfo mutedUser = 4;
|
||||
uint32 mutedSeconds = 5;
|
||||
}
|
||||
|
||||
message GroupMemberCancelMutedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
int64 operationTime = 3;
|
||||
GroupMemberFullInfo mutedUser = 4;
|
||||
}
|
||||
|
||||
message GroupMutedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
int64 operationTime = 3;
|
||||
}
|
||||
|
||||
message GroupCancelMutedTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
int64 operationTime = 3;
|
||||
}
|
||||
|
||||
message GroupMemberInfoSetTips{
|
||||
GroupInfo group = 1;
|
||||
GroupMemberFullInfo opUser = 2;
|
||||
int64 operationTime = 3;
|
||||
GroupMemberFullInfo changedUser = 4;
|
||||
}
|
||||
|
||||
|
||||
message OrganizationChangedTips{
|
||||
UserInfo opUser = 2;
|
||||
int64 operationTime = 3;
|
||||
}
|
||||
|
||||
|
||||
//////////////////////friend/////////////////////
|
||||
//message FriendInfo{
|
||||
// UserInfo OwnerUser = 1;
|
||||
// string Remark = 2;
|
||||
// uint64 CreateTime = 3;
|
||||
// UserInfo FriendUser = 4;
|
||||
//}
|
||||
|
||||
message FriendApplication{
|
||||
int64 addTime = 1;
|
||||
string addSource = 2;
|
||||
string addWording = 3;
|
||||
}
|
||||
|
||||
message FromToUserID{
|
||||
string fromUserID = 1;
|
||||
string toUserID = 2;
|
||||
}
|
||||
|
||||
//FromUserID apply to add ToUserID
|
||||
message FriendApplicationTips{
|
||||
FromToUserID fromToUserID = 1;
|
||||
}
|
||||
|
||||
//FromUserID accept or reject ToUserID
|
||||
message FriendApplicationApprovedTips{
|
||||
FromToUserID fromToUserID = 1;
|
||||
string handleMsg = 2;
|
||||
}
|
||||
|
||||
//FromUserID accept or reject ToUserID
|
||||
message FriendApplicationRejectedTips{
|
||||
FromToUserID fromToUserID = 1;
|
||||
string handleMsg = 2;
|
||||
}
|
||||
|
||||
|
||||
// FromUserID Added a friend ToUserID
|
||||
message FriendAddedTips{
|
||||
FriendInfo friend = 1;
|
||||
int64 operationTime = 2;
|
||||
PublicUserInfo opUser = 3; //who do this
|
||||
|
||||
}
|
||||
|
||||
// FromUserID deleted a friend ToUserID
|
||||
message FriendDeletedTips{
|
||||
FromToUserID fromToUserID = 1;
|
||||
}
|
||||
|
||||
|
||||
|
||||
message BlackAddedTips{
|
||||
FromToUserID fromToUserID = 1;
|
||||
}
|
||||
|
||||
message BlackDeletedTips{
|
||||
FromToUserID fromToUserID = 1;
|
||||
}
|
||||
|
||||
message FriendInfoChangedTips{
|
||||
FromToUserID fromToUserID = 1;
|
||||
}
|
||||
//////////////////////user/////////////////////
|
||||
message UserInfoUpdatedTips{
|
||||
string userID = 1;
|
||||
}
|
||||
|
||||
//////////////////////conversation/////////////////////
|
||||
message ConversationUpdateTips{
|
||||
string UserID = 1;
|
||||
repeated string conversationIDList = 2;
|
||||
int64 updateUnreadCountTime = 3;
|
||||
|
||||
|
||||
}
|
||||
|
||||
message ConversationSetPrivateTips{
|
||||
string recvID = 1;
|
||||
string sendID = 2;
|
||||
bool isPrivate = 3;
|
||||
}
|
||||
|
||||
////////////////////message///////////////////////
|
||||
message DeleteMessageTips{
|
||||
string opUserID = 1;
|
||||
string userID = 2;
|
||||
repeated uint32 seqList = 3;
|
||||
}
|
||||
///cms
|
||||
message RequestPagination {
|
||||
int32 pageNumber = 1;
|
||||
int32 showNumber = 2;
|
||||
}
|
||||
|
||||
message ResponsePagination {
|
||||
int32 CurrentPage = 5;
|
||||
int32 ShowNumber = 6;
|
||||
}
|
||||
|
||||
|
||||
///////////////////signal//////////////
|
||||
message SignalReq {
|
||||
oneof payload {
|
||||
SignalInviteReq invite = 1;
|
||||
SignalInviteInGroupReq inviteInGroup = 2;
|
||||
SignalCancelReq cancel = 3;
|
||||
SignalAcceptReq accept = 4;
|
||||
SignalHungUpReq hungUp = 5;
|
||||
SignalRejectReq reject = 6;
|
||||
SignalGetRoomByGroupIDReq getRoomByGroupID = 7;
|
||||
|
||||
SignalOnRoomParticipantConnectedReq onRoomParticipantConnectedReq = 8;
|
||||
SignalOnRoomParticipantDisconnectedReq onRoomParticipantDisconnectedReq = 9;
|
||||
SignalGetTokenByRoomIDReq getTokenByRoomID = 10;
|
||||
}
|
||||
}
|
||||
|
||||
message SignalResp {
|
||||
oneof payload {
|
||||
SignalInviteReply invite = 1;
|
||||
SignalInviteInGroupReply inviteInGroup = 2;
|
||||
SignalCancelReply cancel = 3;
|
||||
SignalAcceptReply accept = 4;
|
||||
SignalHungUpReply hungUp = 5;
|
||||
SignalRejectReply reject = 6;
|
||||
SignalGetRoomByGroupIDReply getRoomByGroupID = 7;
|
||||
SignalGetTokenByRoomIDReply getTokenByRoomID = 8;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
message InvitationInfo {
|
||||
string inviterUserID = 1;
|
||||
repeated string inviteeUserIDList = 2;
|
||||
string customData = 3;
|
||||
string groupID = 4;
|
||||
string roomID = 5;
|
||||
int32 timeout = 6;
|
||||
string mediaType = 7;
|
||||
int32 platformID = 8;
|
||||
int32 sessionType = 9;
|
||||
int32 initiateTime = 10;
|
||||
repeated string busyLineUserIDList = 11;
|
||||
}
|
||||
|
||||
message ParticipantMetaData{
|
||||
GroupInfo groupInfo = 1;
|
||||
GroupMemberFullInfo groupMemberInfo = 2;
|
||||
PublicUserInfo userInfo = 3;
|
||||
}
|
||||
|
||||
message SignalInviteReq {
|
||||
string opUserID = 1;
|
||||
InvitationInfo invitation = 2;
|
||||
OfflinePushInfo offlinePushInfo = 3;
|
||||
ParticipantMetaData participant = 4;
|
||||
|
||||
}
|
||||
|
||||
message SignalInviteReply {
|
||||
string token = 1;
|
||||
string roomID = 2;
|
||||
string liveURL = 3;
|
||||
repeated string busyLineUserIDList = 4;
|
||||
}
|
||||
|
||||
message SignalInviteInGroupReq {
|
||||
string opUserID = 1;
|
||||
InvitationInfo invitation = 2;
|
||||
OfflinePushInfo offlinePushInfo = 3;
|
||||
ParticipantMetaData participant = 4;
|
||||
}
|
||||
|
||||
message SignalInviteInGroupReply {
|
||||
string token = 1;
|
||||
string roomID = 2;
|
||||
string liveURL = 3;
|
||||
repeated string busyLineUserIDList = 4;
|
||||
}
|
||||
|
||||
message SignalCancelReq {
|
||||
string opUserID = 1;
|
||||
InvitationInfo invitation = 2;
|
||||
OfflinePushInfo offlinePushInfo = 3;
|
||||
ParticipantMetaData participant = 4;
|
||||
}
|
||||
|
||||
message SignalCancelReply {
|
||||
|
||||
}
|
||||
|
||||
message SignalAcceptReq {
|
||||
string opUserID = 1;
|
||||
InvitationInfo invitation = 2;
|
||||
OfflinePushInfo offlinePushInfo = 3;
|
||||
ParticipantMetaData participant = 4;
|
||||
int32 opUserPlatformID = 5;
|
||||
}
|
||||
|
||||
message SignalAcceptReply {
|
||||
string token = 1;
|
||||
string roomID = 2;
|
||||
string liveURL = 3;
|
||||
}
|
||||
|
||||
message SignalHungUpReq {
|
||||
string opUserID = 1;
|
||||
InvitationInfo invitation = 2;
|
||||
OfflinePushInfo offlinePushInfo = 3;
|
||||
}
|
||||
|
||||
message SignalHungUpReply {
|
||||
|
||||
}
|
||||
|
||||
|
||||
message SignalRejectReq {
|
||||
string opUserID = 1;
|
||||
InvitationInfo invitation = 2;
|
||||
OfflinePushInfo offlinePushInfo = 3;
|
||||
ParticipantMetaData participant = 4;
|
||||
int32 opUserPlatformID = 5;
|
||||
}
|
||||
|
||||
message SignalRejectReply {
|
||||
|
||||
}
|
||||
|
||||
message SignalGetRoomByGroupIDReq {
|
||||
string opUserID = 1;
|
||||
string groupID = 2;
|
||||
ParticipantMetaData participant = 3;
|
||||
}
|
||||
|
||||
message SignalGetRoomByGroupIDReply {
|
||||
InvitationInfo invitation = 1;
|
||||
repeated ParticipantMetaData participant = 2;
|
||||
string roomID = 3;
|
||||
}
|
||||
|
||||
message SignalOnRoomParticipantConnectedReq {
|
||||
InvitationInfo invitation = 1;
|
||||
repeated ParticipantMetaData participant = 2;
|
||||
string groupID = 3;
|
||||
}
|
||||
|
||||
message SignalOnRoomParticipantDisconnectedReq {
|
||||
InvitationInfo invitation = 1;
|
||||
repeated ParticipantMetaData participant = 2;
|
||||
string groupID = 3;
|
||||
}
|
||||
|
||||
message SignalGetTokenByRoomIDReq {
|
||||
string roomID = 1;
|
||||
string opUserID = 2;
|
||||
ParticipantMetaData participant = 3;
|
||||
string operationID = 4;
|
||||
}
|
||||
|
||||
message SignalGetTokenByRoomIDReply {
|
||||
string token = 1;
|
||||
string liveURL = 2;
|
||||
}
|
||||
|
||||
|
||||
message DelMsgListReq{
|
||||
string opUserID = 1;
|
||||
string userID = 2;
|
||||
repeated uint32 seqList = 3;
|
||||
string operationID = 4;
|
||||
}
|
||||
|
||||
message DelMsgListResp{
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
}
|
||||
|
||||
message SetAppBackgroundStatusReq {
|
||||
string userID = 1;
|
||||
bool isBackground = 2;
|
||||
}
|
||||
|
||||
message SetAppBackgroundStatusResp {
|
||||
int32 errCode = 1;
|
||||
string errMsg = 2;
|
||||
}
|
||||
|
||||
message ExtendMsgSet {
|
||||
string sourceID = 1;
|
||||
int32 sessionType = 2;
|
||||
map <string, ExtendMsg>extendMsgs = 3;
|
||||
int64 MaxMsgUpdateTime = 4;
|
||||
int32 extendMsgNum = 5;
|
||||
int64 createTime = 6;
|
||||
}
|
||||
|
||||
message ExtendMsg {
|
||||
map <string, KeyValue>reactionExtensionList = 1;
|
||||
string clientMsgID = 2;
|
||||
int64 msgFirstModifyTime = 3;
|
||||
string attachedInfo = 4;
|
||||
string ex = 5;
|
||||
}
|
||||
|
||||
message KeyValue {
|
||||
string typeKey = 1;
|
||||
string value = 2;
|
||||
int64 latestUpdateTime = 3;
|
||||
}
|
||||
|
||||
|
||||
|
||||
@ -1,15 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package data_conversion
|
||||
@ -0,0 +1,104 @@
|
||||
package utils
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/schema"
|
||||
"log"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
)
|
||||
|
||||
func FindAndInsert[V2 any, V3 schema.Tabler](v2db *gorm.DB, v3db *gorm.DB, fn func(V2) (V3, bool)) (string, error) {
|
||||
const batchSize = 100
|
||||
var t V3
|
||||
name := t.TableName()
|
||||
if err := v3db.AutoMigrate(&t); err != nil {
|
||||
return name, fmt.Errorf("auto migrate v3 %s failed %w", name, err)
|
||||
}
|
||||
for i := 0; ; i++ {
|
||||
var v2s []V2
|
||||
if err := v2db.Offset(i * batchSize).Limit(batchSize).Find(&v2s).Error; err != nil {
|
||||
return name, fmt.Errorf("find v2 %s failed %w", name, err)
|
||||
}
|
||||
if len(v2s) == 0 {
|
||||
return name, nil
|
||||
}
|
||||
v3s := make([]V3, 0, len(v2s))
|
||||
for _, v := range v2s {
|
||||
res, ok := fn(v)
|
||||
if ok {
|
||||
v3s = append(v3s, res)
|
||||
}
|
||||
}
|
||||
if len(v3s) == 0 {
|
||||
continue
|
||||
}
|
||||
if err := v3db.Create(&v3s).Error; err != nil {
|
||||
return name, fmt.Errorf("insert v3 %s failed %w", name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type TakeList []Task
|
||||
|
||||
func (l *TakeList) Append(fn ...Task) {
|
||||
*l = append(*l, fn...)
|
||||
}
|
||||
|
||||
type Task func() (string, error)
|
||||
|
||||
func RunTask(concurrency int, tasks TakeList) []string {
|
||||
if len(tasks) == 0 {
|
||||
return []string{}
|
||||
}
|
||||
if concurrency < 1 {
|
||||
concurrency = 1
|
||||
}
|
||||
if concurrency > len(tasks) {
|
||||
concurrency = len(tasks)
|
||||
}
|
||||
|
||||
taskCh := make(chan func() (string, error), 4)
|
||||
go func() {
|
||||
defer close(taskCh)
|
||||
for i := range tasks {
|
||||
taskCh <- tasks[i]
|
||||
}
|
||||
}()
|
||||
|
||||
var lock sync.Mutex
|
||||
var failedTables []string
|
||||
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(concurrency)
|
||||
var count int64
|
||||
|
||||
for i := 0; i < concurrency; i++ {
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for task := range taskCh {
|
||||
name, err := task()
|
||||
index := atomic.AddInt64(&count, 1)
|
||||
if err == nil {
|
||||
log.Printf("[%d/%d] %s success\n", index, len(tasks), name)
|
||||
} else {
|
||||
lock.Lock()
|
||||
failedTables = append(failedTables, name)
|
||||
lock.Unlock()
|
||||
log.Printf("[%d/%d] %s failed %s\n", index, len(tasks), name, err)
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
if len(failedTables) == 0 {
|
||||
log.Println("all tables success")
|
||||
} else {
|
||||
log.Printf("failed tables %d: %+v\n", len(failedTables), failedTables)
|
||||
}
|
||||
|
||||
return failedTables
|
||||
}
|
||||
@ -0,0 +1,14 @@
|
||||
package utils
|
||||
|
||||
import "time"
|
||||
|
||||
func InitTime(ts ...*time.Time) {
|
||||
for i := range ts {
|
||||
if ts[i] == nil {
|
||||
continue
|
||||
}
|
||||
if ts[i].IsZero() || ts[i].UnixMicro() < 0 {
|
||||
*ts[i] = time.UnixMicro(0)
|
||||
}
|
||||
}
|
||||
}
|
||||
Loading…
Reference in new issue