Merge pull request #3014 from openimsdk/cherry-pick-7563aff
deps: Merge #2914 #2957 #2972 #2973 #2977 #2985 #2993 #2995 #3001 #3002 #3007 #3009 PRs into pre-release-v3.8.3pull/3023/head
chao
9 months ago
committed by
GitHub
commit
9a122d2eb3
@ -1,175 +1,188 @@
|
||||
# OpenIM Application Containerization Deployment Guide
|
||||
# Kubernetes Deployment
|
||||
|
||||
OpenIM supports a variety of cluster deployment methods, including but not limited to `helm`, `sealos`, `kustomize`
|
||||
## Resource Requests
|
||||
|
||||
Various contributors, as well as previous official releases, have provided some referenceable solutions:
|
||||
- CPU: 2 cores
|
||||
- Memory: 4 GiB
|
||||
- Disk usage: 20 GiB (on Node)
|
||||
|
||||
+ [k8s-jenkins Repository](https://github.com/OpenIMSDK/k8s-jenkins)
|
||||
+ [open-im-server-k8s-deploy Repository](https://github.com/openimsdk/open-im-server-k8s-deploy)
|
||||
+ [openim-charts Repository](https://github.com/OpenIMSDK/openim-charts)
|
||||
+ [deploy-openim Repository](https://github.com/showurl/deploy-openim)
|
||||
## Preconditions
|
||||
|
||||
### Dependency Check
|
||||
ensure that you have already deployed the following components:
|
||||
|
||||
```bash
|
||||
Kubernetes: >= 1.16.0-0
|
||||
Helm: >= 3.0
|
||||
```
|
||||
- Redis
|
||||
- MongoDB
|
||||
- Kafka
|
||||
- MinIO
|
||||
|
||||
### Minimum Configuration
|
||||
## Origin Deploy
|
||||
|
||||
The recommended minimum configuration for a production environment is as follows:
|
||||
### Enter the target dir
|
||||
|
||||
```yaml
|
||||
CPU: 4
|
||||
Memory: 8G
|
||||
Disk: 100G
|
||||
```
|
||||
`cd ./deployments/deploy/`
|
||||
|
||||
## Configuration File Generation
|
||||
### Deploy configs and dependencies
|
||||
|
||||
We have automated all the files, making the generation of configuration files optional for OpenIM. However, if you desire custom configurations, you can follow the steps below:
|
||||
Upate your configMap `openim-config.yml`. **You can check the official docs for more details.**
|
||||
|
||||
```bash
|
||||
$ make init
|
||||
# Alternatively, use script:
|
||||
# ./scripts/init-config.sh
|
||||
```
|
||||
In `openim-config.yml`, you need modify the following configurations:
|
||||
|
||||
At this point, configuration files will be generated under `deployments/openim/config`, which you can modify as per your requirements.
|
||||
**discovery.yml**
|
||||
|
||||
## Cluster Setup
|
||||
- `kubernetes.namespace`: default is `default`, you can change it to your namespace.
|
||||
|
||||
If you already have a `kubernetes` cluster, or if you wish to build a `kubernetes` cluster from scratch, you can skip this step.
|
||||
**mongodb.yml**
|
||||
|
||||
For a quick start, I used [sealos](https://github.com/labring/sealos) to rapidly set up the cluster, with sealos also being a wrapper for kubeadm at its core:
|
||||
- `address`: set to your already mongodb address or mongo Service name and port in your deployed.
|
||||
- `database`: set to your mongodb database name.(Need have a created database.)
|
||||
- `authSource`: set to your mongodb authSource. (authSource is specify the database name associated with the user's credentials, user need create in this database.)
|
||||
|
||||
```bash
|
||||
$ SEALOS_VERSION=`curl -s https://api.github.com/repos/labring/sealos/releases/latest | grep -oE '"tag_name": "[^"]+"' | head -n1 | cut -d'"' -f4` && \
|
||||
curl -sfL https://raw.githubusercontent.com/labring/sealos/${SEALOS_VERSION}/scripts/install.sh |
|
||||
sh -s ${SEALOS_VERSION} labring/sealos
|
||||
```
|
||||
**kafka.yml**
|
||||
|
||||
**Supported Versions:**
|
||||
- `address`: set to your already kafka address or kafka Service name and port in your deployed.
|
||||
|
||||
+ docker: `labring/kubernetes-docker`:(v1.24.0~v1.27.0)
|
||||
+ containerd: `labring/kubernetes`:(v1.24.0~v1.27.0)
|
||||
**redis.yml**
|
||||
|
||||
#### Cluster Installation:
|
||||
- `address`: set to your already redis address or redis Service name and port in your deployed.
|
||||
|
||||
Cluster details are as follows:
|
||||
**minio.yml**
|
||||
|
||||
| Hostname | IP Address | System Info |
|
||||
| -------- | ---------- | ------------------------------------------------------------ |
|
||||
| master01 | 10.0.0.9 | `Linux VM-0-9-ubuntu 5.15.0-76-generic #83-Ubuntu SMP Thu Jun 15 19:16:32 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux` |
|
||||
| node01 | 10.0.0.4 | Similar to master01 |
|
||||
| node02 | 10.0.0.10 | Similar to master01 |
|
||||
- `internalAddress`: set to your minio Service name and port in your deployed.
|
||||
- `externalAddress`: set to your already expose minio external address.
|
||||
|
||||
```bash
|
||||
$ export CLUSTER_USERNAME=ubuntu
|
||||
$ export CLUSTER_PASSWORD=123456
|
||||
$ sudo sealos run labring/kubernetes:v1.25.0 labring/helm:v3.8.2 labring/calico:v3.24.1 \
|
||||
--masters 10.0.0.9 \
|
||||
--nodes 10.0.0.4,10.0.0.10 \
|
||||
-u "$CLUSTER_USERNAME" \
|
||||
-p "$CLUSTER_PASSWORD"
|
||||
```
|
||||
### Set the secret
|
||||
|
||||
> **Node** Uninstallation method: using `kubeadm` for uninstallation does not remove `etcd` and `cni` related configurations. Manual clearance or using `sealos` for uninstallation is needed.
|
||||
>
|
||||
> ```bash
|
||||
> $ sealos reset
|
||||
> ```
|
||||
A Secret is an object that contains a small amount of sensitive data. Such as password and secret. Secret is similar to ConfigMaps.
|
||||
|
||||
If you are local, you can also use Kind and Minikube to test, for example, using Kind:
|
||||
#### Redis:
|
||||
|
||||
```bash
|
||||
$ GO111MODULE="on" go get sigs.k8s.io/kind@v0.11.1
|
||||
$ kind create cluster
|
||||
```
|
||||
Update the `redis-password` value in `redis-secret.yml` to your Redis password encoded in base64.
|
||||
|
||||
### Installing helm
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-redis-secret
|
||||
type: Opaque
|
||||
data:
|
||||
redis-password: b3BlbklNMTIz # update to your redis password encoded in base64, if need empty, you can set to ""
|
||||
```
|
||||
|
||||
Helm simplifies the deployment and management of Kubernetes applications to a large extent by offering version control and release management through packaging.
|
||||
#### Mongo:
|
||||
|
||||
**Using Script:**
|
||||
Update the `mongo_openim_username`, `mongo_openim_password` value in `mongo-secret.yml` to your Mongo username and password encoded in base64.
|
||||
|
||||
```bash
|
||||
$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-mongo-secret
|
||||
type: Opaque
|
||||
data:
|
||||
mongo_openim_username: b3BlbklN # update to your mongo username encoded in base64, if need empty, you can set to "" (this user credentials need in authSource database).
|
||||
mongo_openim_password: b3BlbklNMTIz # update to your mongo password encoded in base64, if need empty, you can set to ""
|
||||
```
|
||||
|
||||
**Adding Repository:**
|
||||
#### Minio:
|
||||
|
||||
```bash
|
||||
$ helm repo add brigade https://openimsdk.github.io/openim-charts
|
||||
Update the `minio-root-user` and `minio-root-password` value in `minio-secret.yml` to your MinIO accessKeyID and secretAccessKey encoded in base64.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-minio-secret
|
||||
type: Opaque
|
||||
data:
|
||||
minio-root-user: cm9vdA== # update to your minio accessKeyID encoded in base64, if need empty, you can set to ""
|
||||
minio-root-password: b3BlbklNMTIz # update to your minio secretAccessKey encoded in base64, if need empty, you can set to ""
|
||||
```
|
||||
|
||||
### OpenIM Image Strategy
|
||||
#### Kafka:
|
||||
|
||||
Automated offerings include aliyun, ghcr, docker hub: [Image Documentation](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/images.md)
|
||||
Update the `kafka-password` value in `kafka-secret.yml` to your Kafka password encoded in base64.
|
||||
|
||||
**Local Test Build Method:**
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-kafka-secret
|
||||
type: Opaque
|
||||
data:
|
||||
kafka-password: b3BlbklNMTIz # update to your kafka password encoded in base64, if need empty, you can set to ""
|
||||
```
|
||||
|
||||
```bash
|
||||
$ make image
|
||||
### Apply the secret.
|
||||
|
||||
```shell
|
||||
kubectl apply -f redis-secret.yml -f minio-secret.yml -f mongo-secret.yml -f kafka-secret.yml
|
||||
```
|
||||
|
||||
> This command assists in quickly building the required images locally. For a detailed build strategy, refer to the [Build Documentation](https://github.com/openimsdk/open-im-server/blob/main/build/README.md).
|
||||
### Apply all config
|
||||
|
||||
## Installation
|
||||
`kubectl apply -f ./openim-config.yml`
|
||||
|
||||
Explore our Helm-Charts repository and read through: [Helm-Charts Repository](https://github.com/openimsdk/helm-charts)
|
||||
> Attation: If you use `default` namespace, you can excute `clusterRile.yml` to create a cluster role binding for default service account.
|
||||
>
|
||||
> Namespace is modify to `discovery.yml` in `openim-config.yml`, you can change `kubernetes.namespace` to your namespace.
|
||||
|
||||
**Excute `clusterRole.yml`**
|
||||
|
||||
Using the helm charts repository, you can ignore the following configuration, but if you want to just use the server and scale on top of it, you can go ahead:
|
||||
`kubectl apply -f ./clusterRole.yml`
|
||||
|
||||
**Use the Helm template to generate the deployment yaml file: `openim-charts.yaml`**
|
||||
### run all deployments and services
|
||||
|
||||
**Gen Image:**
|
||||
> Note: Ensure that infrastructure services like MinIO, Redis, and Kafka are running before deploying the main applications.
|
||||
|
||||
```bash
|
||||
../scripts/genconfig.sh ../scripts/install/environment.sh ./templates/helm-image.yaml > ./charts/generated-configs/helm-image.yaml
|
||||
kubectl apply \
|
||||
-f openim-api-deployment.yml \
|
||||
-f openim-api-service.yml \
|
||||
-f openim-crontask-deployment.yml \
|
||||
-f openim-rpc-user-deployment.yml \
|
||||
-f openim-rpc-user-service.yml \
|
||||
-f openim-msggateway-deployment.yml \
|
||||
-f openim-msggateway-service.yml \
|
||||
-f openim-push-deployment.yml \
|
||||
-f openim-push-service.yml \
|
||||
-f openim-msgtransfer-service.yml \
|
||||
-f openim-msgtransfer-deployment.yml \
|
||||
-f openim-rpc-conversation-deployment.yml \
|
||||
-f openim-rpc-conversation-service.yml \
|
||||
-f openim-rpc-auth-deployment.yml \
|
||||
-f openim-rpc-auth-service.yml \
|
||||
-f openim-rpc-group-deployment.yml \
|
||||
-f openim-rpc-group-service.yml \
|
||||
-f openim-rpc-friend-deployment.yml \
|
||||
-f openim-rpc-friend-service.yml \
|
||||
-f openim-rpc-msg-deployment.yml \
|
||||
-f openim-rpc-msg-service.yml \
|
||||
-f openim-rpc-third-deployment.yml \
|
||||
-f openim-rpc-third-service.yml
|
||||
```
|
||||
|
||||
**Gen Charts:**
|
||||
### Verification
|
||||
|
||||
After deploying the services, verify that everything is running smoothly:
|
||||
|
||||
```bash
|
||||
for chart in ./charts/*/; do
|
||||
if [[ "$chart" == *"generated-configs"* || "$chart" == *"helmfile.yaml"* ]]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ -f "${chart}values.yaml" ]; then
|
||||
helm template "$chart" -f "./charts/generated-configs/helm-image.yaml" -f "./charts/generated-configs/config.yaml" -f "./charts/generated-configs/notification.yaml" >> openim-charts.yaml
|
||||
else
|
||||
helm template "$chart" >> openim-charts.yaml
|
||||
fi
|
||||
done
|
||||
```
|
||||
# Check the status of all pods
|
||||
kubectl get pods
|
||||
|
||||
**Use Helmfile:**
|
||||
# Check the status of services
|
||||
kubectl get svc
|
||||
|
||||
```bash
|
||||
GO111MODULE=on go get github.com/roboll/helmfile@latest
|
||||
```
|
||||
# Check the status of deployments
|
||||
kubectl get deployments
|
||||
|
||||
```bash
|
||||
export MONGO_ADDRESS=im-mongo
|
||||
export MONGO_PORT=27017
|
||||
export REDIS_ADDRESS=im-redis-master
|
||||
export REDIS_PORT=6379
|
||||
export KAFKA_ADDRESS=im-kafka
|
||||
export KAFKA_PORT=9092
|
||||
export OBJECT_APIURL="https://openim.server.com/api"
|
||||
export MINIO_ENDPOINT="http://im-minio:9000"
|
||||
export MINIO_SIGN_ENDPOINT="https://openim.server.com/im-minio-api"
|
||||
|
||||
mkdir ./charts/generated-configs
|
||||
../scripts/genconfig.sh ../scripts/install/environment.sh ./templates/config.yaml > ./charts/generated-configs/config.yaml
|
||||
cp ../config/notification.yaml ./charts/generated-configs/notification.yaml
|
||||
../scripts/genconfig.sh ../scripts/install/environment.sh ./templates/helm-image.yaml > ./charts/generated-configs/helm-image.yaml
|
||||
# View all resources
|
||||
kubectl get all
|
||||
```
|
||||
|
||||
```bash
|
||||
helmfile apply
|
||||
```
|
||||
### clean all
|
||||
|
||||
`kubectl delete -f ./`
|
||||
|
||||
### Notes:
|
||||
|
||||
- If you use a specific namespace for your deployment, be sure to append the -n <namespace> flag to your kubectl commands.
|
||||
|
||||
@ -0,0 +1,7 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-kafka-secret
|
||||
type: Opaque
|
||||
data:
|
||||
kafka-password: ""
|
||||
@ -0,0 +1,8 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-minio-secret
|
||||
type: Opaque
|
||||
data:
|
||||
minio-root-user: cm9vdA== # Base64 encoded "root"
|
||||
minio-root-password: b3BlbklNMTIz # Base64 encoded "openIM123"
|
||||
@ -0,0 +1,79 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: minio
|
||||
labels:
|
||||
app: minio
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: minio
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: minio
|
||||
spec:
|
||||
containers:
|
||||
- name: minio
|
||||
image: minio/minio:RELEASE.2024-01-11T07-46-16Z
|
||||
ports:
|
||||
- containerPort: 9000 # MinIO service port
|
||||
- containerPort: 9090 # MinIO console port
|
||||
volumeMounts:
|
||||
- name: minio-data
|
||||
mountPath: /data
|
||||
- name: minio-config
|
||||
mountPath: /root/.minio
|
||||
env:
|
||||
- name: TZ
|
||||
value: "Asia/Shanghai"
|
||||
- name: MINIO_ROOT_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-minio-secret
|
||||
key: minio-root-user
|
||||
- name: MINIO_ROOT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-minio-secret
|
||||
key: minio-root-password
|
||||
command:
|
||||
- "/bin/sh"
|
||||
- "-c"
|
||||
- |
|
||||
mkdir -p /data && \
|
||||
minio server /data --console-address ":9090"
|
||||
volumes:
|
||||
- name: minio-data
|
||||
persistentVolumeClaim:
|
||||
claimName: minio-pvc
|
||||
- name: minio-config
|
||||
persistentVolumeClaim:
|
||||
claimName: minio-config-pvc
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: minio-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: minio-config-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 2Gi
|
||||
|
||||
|
||||
@ -0,0 +1,8 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-mongo-secret
|
||||
type: Opaque
|
||||
data:
|
||||
mongo_openim_username: b3BlbklN # base64 for "openIM", this user credentials need in authSource database.
|
||||
mongo_openim_password: b3BlbklNMTIz # base64 for "openIM123"
|
||||
@ -0,0 +1,108 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: mongo-statefulset
|
||||
spec:
|
||||
serviceName: "mongo"
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: mongo
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: mongo
|
||||
spec:
|
||||
containers:
|
||||
- name: mongo
|
||||
image: mongo:7.0
|
||||
command: ["/bin/bash", "-c"]
|
||||
args:
|
||||
- >
|
||||
docker-entrypoint.sh mongod --wiredTigerCacheSizeGB ${wiredTigerCacheSizeGB} --auth &
|
||||
until mongosh -u ${MONGO_INITDB_ROOT_USERNAME} -p ${MONGO_INITDB_ROOT_PASSWORD} --authenticationDatabase admin --eval "db.runCommand({ ping: 1 })" &>/dev/null; do
|
||||
echo "Waiting for MongoDB to start...";
|
||||
sleep 1;
|
||||
done &&
|
||||
mongosh -u ${MONGO_INITDB_ROOT_USERNAME} -p ${MONGO_INITDB_ROOT_PASSWORD} --authenticationDatabase admin --eval "
|
||||
db = db.getSiblingDB(\"${MONGO_INITDB_DATABASE}\");
|
||||
if (!db.getUser(\"${MONGO_OPENIM_USERNAME}\")) {
|
||||
db.createUser({
|
||||
user: \"${MONGO_OPENIM_USERNAME}\",
|
||||
pwd: \"${MONGO_OPENIM_PASSWORD}\",
|
||||
roles: [{role: \"readWrite\", db: \"${MONGO_INITDB_DATABASE}\"}]
|
||||
});
|
||||
print(\"User created successfully: \");
|
||||
print(\"Username: ${MONGO_OPENIM_USERNAME}\");
|
||||
print(\"Password: ${MONGO_OPENIM_PASSWORD}\");
|
||||
print(\"Database: ${MONGO_INITDB_DATABASE}\");
|
||||
} else {
|
||||
print(\"User already exists in database: ${MONGO_INITDB_DATABASE}, Username: ${MONGO_OPENIM_USERNAME}\");
|
||||
}
|
||||
" &&
|
||||
tail -f /dev/null
|
||||
ports:
|
||||
- containerPort: 27017
|
||||
env:
|
||||
- name: MONGO_INITDB_ROOT_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-init-secret
|
||||
key: mongo_initdb_root_username
|
||||
- name: MONGO_INITDB_ROOT_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-init-secret
|
||||
key: mongo_initdb_root_password
|
||||
- name: MONGO_INITDB_DATABASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-init-secret
|
||||
key: mongo_initdb_database
|
||||
- name: MONGO_OPENIM_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-init-secret
|
||||
key: mongo_openim_username
|
||||
- name: MONGO_OPENIM_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-init-secret
|
||||
key: mongo_openim_password
|
||||
- name: TZ
|
||||
value: "Asia/Shanghai"
|
||||
- name: wiredTigerCacheSizeGB
|
||||
value: "1"
|
||||
volumeMounts:
|
||||
- name: mongo-storage
|
||||
mountPath: /data/db
|
||||
|
||||
volumes:
|
||||
- name: mongo-storage
|
||||
persistentVolumeClaim:
|
||||
claimName: mongo-pvc
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: mongo-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 5Gi
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-mongo-init-secret
|
||||
type: Opaque
|
||||
data:
|
||||
mongo_initdb_root_username: cm9vdA== # base64 for "root"
|
||||
mongo_initdb_root_password: b3BlbklNMTIz # base64 for "openIM123"
|
||||
mongo_initdb_database: b3BlbmltX3Yz # base64 for "openim_v3"
|
||||
mongo_openim_username: b3BlbklN # base64 for "openIM"
|
||||
mongo_openim_password: b3BlbklNMTIz # base64 for "openIM123"
|
||||
@ -0,0 +1,47 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: openim-api
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: openim-api
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: openim-api
|
||||
spec:
|
||||
containers:
|
||||
- name: openim-api-container
|
||||
image: openim/openim-api:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_MONGODB_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_username
|
||||
- name: IMENV_MONGODB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_password
|
||||
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10002
|
||||
- containerPort: 12002
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,36 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: messagegateway-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: messagegateway-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: messagegateway-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: openim-msggateway-container
|
||||
image: openim/openim-msggateway:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10140
|
||||
- containerPort: 12001
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,50 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: openim-msgtransfer-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: openim-msgtransfer-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: openim-msgtransfer-server
|
||||
spec:
|
||||
containers:
|
||||
- name: openim-msgtransfer-container
|
||||
image: openim/openim-msgtransfer:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_MONGODB_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_username
|
||||
- name: IMENV_MONGODB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_password
|
||||
- name: IMENV_KAFKA_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-kafka-secret
|
||||
key: kafka-password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 12020
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,41 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: push-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: push-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: push-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: push-rpc-server-container
|
||||
image: openim/openim-push:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_KAFKA_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-kafka-secret
|
||||
key: kafka-password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10170
|
||||
- containerPort: 12170
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,37 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: auth-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: auth-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: auth-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: auth-rpc-server-container
|
||||
image: openim/openim-rpc-auth:v3.8.3
|
||||
imagePullPolicy: Never
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10200
|
||||
- containerPort: 12200
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,46 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: conversation-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: conversation-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: conversation-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: conversation-rpc-server-container
|
||||
image: openim/openim-rpc-conversation:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_MONGODB_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_username
|
||||
- name: IMENV_MONGODB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10220
|
||||
- containerPort: 12220
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,46 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: friend-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: friend-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: friend-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: friend-rpc-server-container
|
||||
image: openim/openim-rpc-friend:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_MONGODB_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_username
|
||||
- name: IMENV_MONGODB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10240
|
||||
- containerPort: 12240
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,46 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: group-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: group-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: group-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: group-rpc-server-container
|
||||
image: openim/openim-rpc-group:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_MONGODB_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_username
|
||||
- name: IMENV_MONGODB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10260
|
||||
- containerPort: 12260
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,51 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: msg-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: msg-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: msg-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: msg-rpc-server-container
|
||||
image: openim/openim-rpc-msg:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_MONGODB_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_username
|
||||
- name: IMENV_MONGODB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_password
|
||||
- name: IMENV_KAFKA_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-kafka-secret
|
||||
key: kafka-password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10280
|
||||
- containerPort: 12280
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,56 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: third-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: third-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: third-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: third-rpc-server-container
|
||||
image: openim/openim-rpc-third:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_MINIO_ACCESSKEYID
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-minio-secret
|
||||
key: minio-root-user
|
||||
- name: IMENV_MINIO_SECRETACCESSKEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-minio-secret
|
||||
key: minio-root-password
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_MONGODB_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_username
|
||||
- name: IMENV_MONGODB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10300
|
||||
- containerPort: 12300
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,51 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: user-rpc-server
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: user-rpc-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: user-rpc-server
|
||||
spec:
|
||||
containers:
|
||||
- name: user-rpc-server-container
|
||||
image: openim/openim-rpc-user:v3.8.3
|
||||
env:
|
||||
- name: CONFIG_PATH
|
||||
value: "/config"
|
||||
- name: IMENV_REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-redis-secret
|
||||
key: redis-password
|
||||
- name: IMENV_MONGODB_USERNAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_username
|
||||
- name: IMENV_MONGODB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-mongo-secret
|
||||
key: mongo_openim_password
|
||||
- name: IMENV_KAFKA_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: openim-kafka-secret
|
||||
key: kafka-password
|
||||
volumeMounts:
|
||||
- name: openim-config
|
||||
mountPath: "/config"
|
||||
readOnly: true
|
||||
ports:
|
||||
- containerPort: 10320
|
||||
- containerPort: 12320
|
||||
volumes:
|
||||
- name: openim-config
|
||||
configMap:
|
||||
name: openim-config
|
||||
@ -0,0 +1,7 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: openim-redis-secret
|
||||
type: Opaque
|
||||
data:
|
||||
redis-password: b3BlbklNMTIz # "openIM123" in base64
|
||||
@ -0,0 +1,55 @@
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: redis-statefulset
|
||||
spec:
|
||||
serviceName: "redis"
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: redis
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: redis
|
||||
spec:
|
||||
containers:
|
||||
- name: redis
|
||||
image: redis:7.0.0
|
||||
ports:
|
||||
- containerPort: 6379
|
||||
env:
|
||||
- name: TZ
|
||||
value: "Asia/Shanghai"
|
||||
- name: REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-secret
|
||||
key: redis-password
|
||||
volumeMounts:
|
||||
- name: redis-data
|
||||
mountPath: /data
|
||||
command:
|
||||
[
|
||||
"/bin/sh",
|
||||
"-c",
|
||||
'redis-server --requirepass "$REDIS_PASSWORD" --appendonly yes',
|
||||
]
|
||||
volumes:
|
||||
- name: redis-config-volume
|
||||
configMap:
|
||||
name: openim-config
|
||||
- name: redis-data
|
||||
persistentVolumeClaim:
|
||||
claimName: redis-pvc
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: redis-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 5Gi
|
||||
@ -1,32 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package api
|
||||
|
||||
import (
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/openimsdk/open-im-server/v3/pkg/rpcclient"
|
||||
"github.com/openimsdk/protocol/user"
|
||||
"github.com/openimsdk/tools/a2r"
|
||||
)
|
||||
|
||||
type StatisticsApi rpcclient.User
|
||||
|
||||
func NewStatisticsApi(client rpcclient.User) StatisticsApi {
|
||||
return StatisticsApi(client)
|
||||
}
|
||||
|
||||
func (s *StatisticsApi) UserRegister(c *gin.Context) {
|
||||
a2r.Call(user.UserClient.UserRegisterCount, s.Client, c)
|
||||
}
|
||||
@ -0,0 +1,63 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"context"
|
||||
"github.com/openimsdk/open-im-server/v3/pkg/common/config"
|
||||
kdisc "github.com/openimsdk/open-im-server/v3/pkg/common/discoveryregister"
|
||||
pbconversation "github.com/openimsdk/protocol/conversation"
|
||||
"github.com/openimsdk/protocol/msg"
|
||||
"github.com/openimsdk/protocol/third"
|
||||
"github.com/openimsdk/tools/mcontext"
|
||||
"github.com/openimsdk/tools/mw"
|
||||
"github.com/robfig/cron/v3"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials/insecure"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestName(t *testing.T) {
|
||||
conf := &config.Discovery{
|
||||
Enable: config.ETCD,
|
||||
Etcd: config.Etcd{
|
||||
RootDirectory: "openim",
|
||||
Address: []string{"localhost:12379"},
|
||||
},
|
||||
}
|
||||
client, err := kdisc.NewDiscoveryRegister(conf, "source")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
client.AddOption(mw.GrpcClient(), grpc.WithTransportCredentials(insecure.NewCredentials()))
|
||||
ctx := mcontext.SetOpUserID(context.Background(), "imAdmin")
|
||||
msgConn, err := client.GetConn(ctx, "msg-rpc-service")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
thirdConn, err := client.GetConn(ctx, "third-rpc-service")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
conversationConn, err := client.GetConn(ctx, "conversation-rpc-service")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
srv := &cronServer{
|
||||
ctx: ctx,
|
||||
config: &CronTaskConfig{
|
||||
CronTask: config.CronTask{
|
||||
RetainChatRecords: 1,
|
||||
FileExpireTime: 1,
|
||||
DeleteObjectType: []string{"msg-picture", "msg-file", "msg-voice", "msg-video", "msg-video-snapshot", "sdklog", ""},
|
||||
},
|
||||
},
|
||||
cron: cron.New(),
|
||||
msgClient: msg.NewMsgClient(msgConn),
|
||||
conversationClient: pbconversation.NewConversationClient(conversationConn),
|
||||
thirdClient: third.NewThirdClient(thirdConn),
|
||||
}
|
||||
srv.deleteMsg()
|
||||
//srv.clearS3()
|
||||
//srv.clearUserMsg()
|
||||
}
|
||||
@ -0,0 +1,36 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/openimsdk/protocol/msg"
|
||||
"github.com/openimsdk/tools/log"
|
||||
"github.com/openimsdk/tools/mcontext"
|
||||
"os"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (c *cronServer) deleteMsg() {
|
||||
now := time.Now()
|
||||
deltime := now.Add(-time.Hour * 24 * time.Duration(c.config.CronTask.RetainChatRecords))
|
||||
operationID := fmt.Sprintf("cron_msg_%d_%d", os.Getpid(), deltime.UnixMilli())
|
||||
ctx := mcontext.SetOperationID(c.ctx, operationID)
|
||||
log.ZDebug(ctx, "Destruct chat records", "deltime", deltime, "timestamp", deltime.UnixMilli())
|
||||
const (
|
||||
deleteCount = 10000
|
||||
deleteLimit = 50
|
||||
)
|
||||
var count int
|
||||
for i := 1; i <= deleteCount; i++ {
|
||||
ctx := mcontext.SetOperationID(c.ctx, fmt.Sprintf("%s_%d", operationID, i))
|
||||
resp, err := c.msgClient.DestructMsgs(ctx, &msg.DestructMsgsReq{Timestamp: deltime.UnixMilli(), Limit: deleteLimit})
|
||||
if err != nil {
|
||||
log.ZError(ctx, "cron destruct chat records failed", err)
|
||||
break
|
||||
}
|
||||
count += int(resp.Count)
|
||||
if resp.Count < deleteLimit {
|
||||
break
|
||||
}
|
||||
}
|
||||
log.ZDebug(ctx, "cron destruct chat records end", "deltime", deltime, "cont", time.Since(now), "count", count)
|
||||
}
|
||||
@ -0,0 +1,79 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/openimsdk/protocol/third"
|
||||
"github.com/openimsdk/tools/log"
|
||||
"github.com/openimsdk/tools/mcontext"
|
||||
"os"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (c *cronServer) clearS3() {
|
||||
start := time.Now()
|
||||
deleteTime := start.Add(-time.Hour * 24 * time.Duration(c.config.CronTask.FileExpireTime))
|
||||
operationID := fmt.Sprintf("cron_s3_%d_%d", os.Getpid(), deleteTime.UnixMilli())
|
||||
ctx := mcontext.SetOperationID(c.ctx, operationID)
|
||||
log.ZDebug(ctx, "deleteoutDatedData", "deletetime", deleteTime, "timestamp", deleteTime.UnixMilli())
|
||||
const (
|
||||
deleteCount = 10000
|
||||
deleteLimit = 100
|
||||
)
|
||||
|
||||
var count int
|
||||
for i := 1; i <= deleteCount; i++ {
|
||||
resp, err := c.thirdClient.DeleteOutdatedData(ctx, &third.DeleteOutdatedDataReq{ExpireTime: deleteTime.UnixMilli(), ObjectGroup: c.config.CronTask.DeleteObjectType, Limit: deleteLimit})
|
||||
if err != nil {
|
||||
log.ZError(ctx, "cron deleteoutDatedData failed", err)
|
||||
return
|
||||
}
|
||||
count += int(resp.Count)
|
||||
if resp.Count < deleteLimit {
|
||||
break
|
||||
}
|
||||
}
|
||||
log.ZDebug(ctx, "cron deleteoutDatedData success", "deltime", deleteTime, "cont", time.Since(start), "count", count)
|
||||
}
|
||||
|
||||
// var req *third.DeleteOutdatedDataReq
|
||||
// count1, err := ExtractField(ctx, c.thirdClient.DeleteOutdatedData, req, (*third.DeleteOutdatedDataResp).GetCount)
|
||||
//
|
||||
// c.thirdClient.DeleteOutdatedData(ctx, &third.DeleteOutdatedDataReq{})
|
||||
// msggateway.GetUsersOnlineStatusCaller.Invoke(ctx, &msggateway.GetUsersOnlineStatusReq{})
|
||||
//
|
||||
// var cli ThirdClient
|
||||
//
|
||||
// c111, err := cli.DeleteOutdatedData(ctx, 100)
|
||||
//
|
||||
// cli.ThirdClient.DeleteOutdatedData(ctx, &third.DeleteOutdatedDataReq{})
|
||||
//
|
||||
// cli.AuthSign(ctx, &third.AuthSignReq{})
|
||||
//
|
||||
// cli.SetAppBadge()
|
||||
//
|
||||
//}
|
||||
//
|
||||
//func extractField[A, B, C any](ctx context.Context, fn func(ctx context.Context, req *A, opts ...grpc.CallOption) (*B, error), req *A, get func(*B) C) (C, error) {
|
||||
// resp, err := fn(ctx, req)
|
||||
// if err != nil {
|
||||
// var c C
|
||||
// return c, err
|
||||
// }
|
||||
// return get(resp), nil
|
||||
//}
|
||||
//
|
||||
//func ignore(_ any, err error) error {
|
||||
// return err
|
||||
//}
|
||||
//
|
||||
//type ThirdClient struct {
|
||||
// third.ThirdClient
|
||||
//}
|
||||
//
|
||||
//func (c *ThirdClient) DeleteOutdatedData(ctx context.Context, expireTime int64) (int32, error) {
|
||||
// return extractField(ctx, c.ThirdClient.DeleteOutdatedData, &third.DeleteOutdatedDataReq{ExpireTime: expireTime}, (*third.DeleteOutdatedDataResp).GetCount)
|
||||
//}
|
||||
//
|
||||
//func (c *ThirdClient) DeleteOutdatedData1(ctx context.Context, expireTime int64) error {
|
||||
// return ignore(c.ThirdClient.DeleteOutdatedData(ctx, &third.DeleteOutdatedDataReq{ExpireTime: expireTime}))
|
||||
//}
|
||||
@ -0,0 +1,34 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
pbconversation "github.com/openimsdk/protocol/conversation"
|
||||
"github.com/openimsdk/tools/log"
|
||||
"github.com/openimsdk/tools/mcontext"
|
||||
"os"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (c *cronServer) clearUserMsg() {
|
||||
now := time.Now()
|
||||
operationID := fmt.Sprintf("cron_user_msg_%d_%d", os.Getpid(), now.UnixMilli())
|
||||
ctx := mcontext.SetOperationID(c.ctx, operationID)
|
||||
log.ZDebug(ctx, "clear user msg cron start")
|
||||
const (
|
||||
deleteCount = 10000
|
||||
deleteLimit = 100
|
||||
)
|
||||
var count int
|
||||
for i := 1; i <= deleteCount; i++ {
|
||||
resp, err := c.conversationClient.ClearUserConversationMsg(ctx, &pbconversation.ClearUserConversationMsgReq{Timestamp: now.UnixMilli(), Limit: deleteLimit})
|
||||
if err != nil {
|
||||
log.ZError(ctx, "ClearUserConversationMsg failed.", err)
|
||||
return
|
||||
}
|
||||
count += int(resp.Count)
|
||||
if resp.Count < deleteLimit {
|
||||
break
|
||||
}
|
||||
}
|
||||
log.ZDebug(ctx, "clear user msg cron task completed", "cont", time.Since(now), "count", count)
|
||||
}
|
||||
@ -1,133 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"github.com/openimsdk/protocol/sdkws"
|
||||
"github.com/redis/go-redis/v9"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"google.golang.org/protobuf/proto"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func Test_msgCache_SetMessagesToCache(t *testing.T) {
|
||||
type fields struct {
|
||||
rdb redis.UniversalClient
|
||||
}
|
||||
type args struct {
|
||||
ctx context.Context
|
||||
conversationID string
|
||||
msgs []*sdkws.MsgData
|
||||
}
|
||||
tests := []struct {
|
||||
name string
|
||||
fields fields
|
||||
args args
|
||||
want int
|
||||
wantErr assert.ErrorAssertionFunc
|
||||
}{
|
||||
{"test1", fields{rdb: redis.NewClient(&redis.Options{Addr: "localhost:16379", Username: "", Password: "openIM123", DB: 0})}, args{context.Background(),
|
||||
"cid", []*sdkws.MsgData{{Seq: 1}, {Seq: 2}, {Seq: 3}}}, 3, assert.NoError},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
c := &msgCache{
|
||||
rdb: tt.fields.rdb,
|
||||
}
|
||||
got, err := c.SetMessagesToCache(tt.args.ctx, tt.args.conversationID, tt.args.msgs)
|
||||
if !tt.wantErr(t, err, fmt.Sprintf("SetMessagesToCache(%v, %v, %v)", tt.args.ctx, tt.args.conversationID, tt.args.msgs)) {
|
||||
return
|
||||
}
|
||||
assert.Equalf(t, tt.want, got, "SetMessagesToCache(%v, %v, %v)", tt.args.ctx, tt.args.conversationID, tt.args.msgs)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func Test_msgCache_GetMessagesBySeq(t *testing.T) {
|
||||
type fields struct {
|
||||
rdb redis.UniversalClient
|
||||
}
|
||||
type args struct {
|
||||
ctx context.Context
|
||||
conversationID string
|
||||
seqs []int64
|
||||
}
|
||||
var failedSeq []int64
|
||||
tests := []struct {
|
||||
name string
|
||||
fields fields
|
||||
args args
|
||||
wantSeqMsgs []*sdkws.MsgData
|
||||
wantFailedSeqs []int64
|
||||
wantErr assert.ErrorAssertionFunc
|
||||
}{
|
||||
{"test1", fields{rdb: redis.NewClient(&redis.Options{Addr: "localhost:16379", Password: "openIM123", DB: 0})},
|
||||
args{context.Background(), "cid", []int64{1, 2, 3}},
|
||||
[]*sdkws.MsgData{{Seq: 1}, {Seq: 2}, {Seq: 3}}, failedSeq, assert.NoError},
|
||||
{"test2", fields{rdb: redis.NewClient(&redis.Options{Addr: "localhost:16379", Password: "openIM123", DB: 0})},
|
||||
args{context.Background(), "cid", []int64{4, 5, 6}},
|
||||
nil, []int64{4, 5, 6}, assert.NoError},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
c := &msgCache{
|
||||
rdb: tt.fields.rdb,
|
||||
}
|
||||
gotSeqMsgs, gotFailedSeqs, err := c.GetMessagesBySeq(tt.args.ctx, tt.args.conversationID, tt.args.seqs)
|
||||
if !tt.wantErr(t, err, fmt.Sprintf("GetMessagesBySeq(%v, %v, %v)", tt.args.ctx, tt.args.conversationID, tt.args.seqs)) {
|
||||
return
|
||||
}
|
||||
equalMsgDataSlices(t, tt.wantSeqMsgs, gotSeqMsgs)
|
||||
assert.Equalf(t, tt.wantFailedSeqs, gotFailedSeqs, "GetMessagesBySeq(%v, %v, %v)", tt.args.ctx, tt.args.conversationID, tt.args.seqs)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func equalMsgDataSlices(t *testing.T, expected, actual []*sdkws.MsgData) {
|
||||
assert.Equal(t, len(expected), len(actual), "Slices have different lengths")
|
||||
for i := range expected {
|
||||
assert.True(t, proto.Equal(expected[i], actual[i]), "Element %d not equal: expected %v, got %v", i, expected[i], actual[i])
|
||||
}
|
||||
}
|
||||
|
||||
func Test_msgCache_DeleteMessagesFromCache(t *testing.T) {
|
||||
type fields struct {
|
||||
rdb redis.UniversalClient
|
||||
}
|
||||
type args struct {
|
||||
ctx context.Context
|
||||
conversationID string
|
||||
seqs []int64
|
||||
}
|
||||
tests := []struct {
|
||||
name string
|
||||
fields fields
|
||||
args args
|
||||
wantErr assert.ErrorAssertionFunc
|
||||
}{
|
||||
{"test1", fields{rdb: redis.NewClient(&redis.Options{Addr: "localhost:16379", Password: "openIM123"})},
|
||||
args{context.Background(), "cid", []int64{1, 2, 3}}, assert.NoError},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
c := &msgCache{
|
||||
rdb: tt.fields.rdb,
|
||||
}
|
||||
tt.wantErr(t, c.DeleteMessagesFromCache(tt.args.ctx, tt.args.conversationID, tt.args.seqs),
|
||||
fmt.Sprintf("DeleteMessagesFromCache(%v, %v, %v)", tt.args.ctx, tt.args.conversationID, tt.args.seqs))
|
||||
})
|
||||
}
|
||||
}
|
||||
@ -1,74 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package rpcclient
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/openimsdk/protocol/auth"
|
||||
"github.com/openimsdk/tools/discovery"
|
||||
"github.com/openimsdk/tools/system/program"
|
||||
"google.golang.org/grpc"
|
||||
)
|
||||
|
||||
func NewAuth(discov discovery.SvcDiscoveryRegistry, rpcRegisterName string) *Auth {
|
||||
conn, err := discov.GetConn(context.Background(), rpcRegisterName)
|
||||
if err != nil {
|
||||
program.ExitWithError(err)
|
||||
}
|
||||
client := auth.NewAuthClient(conn)
|
||||
return &Auth{discov: discov, conn: conn, Client: client}
|
||||
}
|
||||
|
||||
type Auth struct {
|
||||
conn grpc.ClientConnInterface
|
||||
Client auth.AuthClient
|
||||
discov discovery.SvcDiscoveryRegistry
|
||||
}
|
||||
|
||||
func (a *Auth) ParseToken(ctx context.Context, token string) (*auth.ParseTokenResp, error) {
|
||||
req := auth.ParseTokenReq{
|
||||
Token: token,
|
||||
}
|
||||
resp, err := a.Client.ParseToken(ctx, &req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
func (a *Auth) InvalidateToken(ctx context.Context, preservedToken, userID string, platformID int) (*auth.InvalidateTokenResp, error) {
|
||||
req := auth.InvalidateTokenReq{
|
||||
PreservedToken: preservedToken,
|
||||
UserID: userID,
|
||||
PlatformID: int32(platformID),
|
||||
}
|
||||
resp, err := a.Client.InvalidateToken(ctx, &req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
|
||||
func (a *Auth) KickTokens(ctx context.Context, tokens []string) (*auth.KickTokensResp, error) {
|
||||
req := auth.KickTokensReq{
|
||||
Tokens: tokens,
|
||||
}
|
||||
resp, err := a.Client.KickTokens(ctx, &req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp, err
|
||||
}
|
||||
@ -1,161 +0,0 @@
|
||||
// Copyright © 2023 OpenIM. All rights reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package rpcclient
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
pbconversation "github.com/openimsdk/protocol/conversation"
|
||||
"github.com/openimsdk/tools/discovery"
|
||||
"github.com/openimsdk/tools/errs"
|
||||
"github.com/openimsdk/tools/system/program"
|
||||
"google.golang.org/grpc"
|
||||
)
|
||||
|
||||
type Conversation struct {
|
||||
Client pbconversation.ConversationClient
|
||||
conn grpc.ClientConnInterface
|
||||
discov discovery.SvcDiscoveryRegistry
|
||||
}
|
||||
|
||||
func NewConversation(discov discovery.SvcDiscoveryRegistry, rpcRegisterName string) *Conversation {
|
||||
conn, err := discov.GetConn(context.Background(), rpcRegisterName)
|
||||
if err != nil {
|
||||
program.ExitWithError(err)
|
||||
}
|
||||
client := pbconversation.NewConversationClient(conn)
|
||||
return &Conversation{discov: discov, conn: conn, Client: client}
|
||||
}
|
||||
|
||||
type ConversationRpcClient Conversation
|
||||
|
||||
func NewConversationRpcClient(discov discovery.SvcDiscoveryRegistry, rpcRegisterName string) ConversationRpcClient {
|
||||
return ConversationRpcClient(*NewConversation(discov, rpcRegisterName))
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GetSingleConversationRecvMsgOpt(ctx context.Context, userID, conversationID string) (int32, error) {
|
||||
var req pbconversation.GetConversationReq
|
||||
req.OwnerUserID = userID
|
||||
req.ConversationID = conversationID
|
||||
conversation, err := c.Client.GetConversation(ctx, &req)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return conversation.GetConversation().RecvMsgOpt, err
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) SingleChatFirstCreateConversation(ctx context.Context, recvID, sendID,
|
||||
conversationID string, conversationType int32) error {
|
||||
_, err := c.Client.CreateSingleChatConversations(ctx,
|
||||
&pbconversation.CreateSingleChatConversationsReq{
|
||||
RecvID: recvID, SendID: sendID, ConversationID: conversationID,
|
||||
ConversationType: conversationType,
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GroupChatFirstCreateConversation(ctx context.Context, groupID string, userIDs []string) error {
|
||||
_, err := c.Client.CreateGroupChatConversations(ctx, &pbconversation.CreateGroupChatConversationsReq{UserIDs: userIDs, GroupID: groupID})
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) SetConversationMaxSeq(ctx context.Context, ownerUserIDs []string, conversationID string, maxSeq int64) error {
|
||||
_, err := c.Client.SetConversationMaxSeq(ctx, &pbconversation.SetConversationMaxSeqReq{OwnerUserID: ownerUserIDs, ConversationID: conversationID, MaxSeq: maxSeq})
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) SetConversationMinSeq(ctx context.Context, ownerUserIDs []string, conversationID string, minSeq int64) error {
|
||||
_, err := c.Client.SetConversationMinSeq(ctx, &pbconversation.SetConversationMinSeqReq{OwnerUserID: ownerUserIDs, ConversationID: conversationID, MinSeq: minSeq})
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) SetConversations(ctx context.Context, userIDs []string, conversation *pbconversation.ConversationReq) error {
|
||||
_, err := c.Client.SetConversations(ctx, &pbconversation.SetConversationsReq{UserIDs: userIDs, Conversation: conversation})
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) UpdateConversation(ctx context.Context, conversation *pbconversation.UpdateConversationReq) error {
|
||||
_, err := c.Client.UpdateConversation(ctx, conversation)
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GetConversationIDs(ctx context.Context, ownerUserID string) ([]string, error) {
|
||||
resp, err := c.Client.GetConversationIDs(ctx, &pbconversation.GetConversationIDsReq{UserID: ownerUserID})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp.ConversationIDs, nil
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GetConversation(ctx context.Context, ownerUserID, conversationID string) (*pbconversation.Conversation, error) {
|
||||
resp, err := c.Client.GetConversation(ctx, &pbconversation.GetConversationReq{OwnerUserID: ownerUserID, ConversationID: conversationID})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp.Conversation, nil
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GetConversationsByConversationID(ctx context.Context, conversationIDs []string) ([]*pbconversation.Conversation, error) {
|
||||
if len(conversationIDs) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
resp, err := c.Client.GetConversationsByConversationID(ctx, &pbconversation.GetConversationsByConversationIDReq{ConversationIDs: conversationIDs})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(resp.Conversations) == 0 {
|
||||
return nil, errs.ErrRecordNotFound.WrapMsg(fmt.Sprintf("conversationIDs: %v not found", conversationIDs))
|
||||
}
|
||||
return resp.Conversations, nil
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GetConversationOfflinePushUserIDs(ctx context.Context, conversationID string, userIDs []string) ([]string, error) {
|
||||
resp, err := c.Client.GetConversationOfflinePushUserIDs(ctx, &pbconversation.GetConversationOfflinePushUserIDsReq{ConversationID: conversationID, UserIDs: userIDs})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp.UserIDs, nil
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GetConversations(ctx context.Context, ownerUserID string, conversationIDs []string) ([]*pbconversation.Conversation, error) {
|
||||
if len(conversationIDs) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
resp, err := c.Client.GetConversations(
|
||||
ctx,
|
||||
&pbconversation.GetConversationsReq{OwnerUserID: ownerUserID, ConversationIDs: conversationIDs},
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp.Conversations, nil
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GetConversationNotReceiveMessageUserIDs(ctx context.Context, conversationID string) ([]string, error) {
|
||||
resp, err := c.Client.GetConversationNotReceiveMessageUserIDs(ctx, &pbconversation.GetConversationNotReceiveMessageUserIDsReq{ConversationID: conversationID})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp.UserIDs, nil
|
||||
}
|
||||
|
||||
func (c *ConversationRpcClient) GetConversationsNeedClearMsg(ctx context.Context) ([]*pbconversation.Conversation, error) {
|
||||
resp, err := c.Client.GetConversationsNeedClearMsg(ctx, &pbconversation.GetConversationsNeedClearMsgReq{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp.Conversations, nil
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue