Date: Wed, 24 Jul 2024 14:50:02 +0800
Subject: [PATCH 05/17] docs: fix english typo (#2432)
---
README.md | 2 +-
README_zh_CN.md | 2 +-
docs/contrib/util-scripts.md | 2 +-
docs/contributing/CONTRIBUTING-JP.md | 2 +-
docs/contributing/CONTRIBUTING-PL.md | 2 +-
docs/readme/README_cs.md | 2 +-
docs/readme/README_da.md | 2 +-
docs/readme/README_el.md | 2 +-
docs/readme/README_es.md | 2 +-
docs/readme/README_fa.md | 2 +-
docs/readme/README_fr.md | 2 +-
docs/readme/README_hu.md | 2 +-
docs/readme/README_ja.md | 2 +-
docs/readme/README_ko.md | 2 +-
docs/readme/README_tr.md | 2 +-
docs/readme/README_uk.md | 2 +-
docs/readme/README_vi.md | 2 +-
17 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/README.md b/README.md
index d73d5749a..a99559cdb 100644
--- a/README.md
+++ b/README.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/README_zh_CN.md b/README_zh_CN.md
index 65aac9ebc..59198eafb 100644
--- a/README_zh_CN.md
+++ b/README_zh_CN.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/contrib/util-scripts.md b/docs/contrib/util-scripts.md
index 0bf6f23e5..30da871a4 100644
--- a/docs/contrib/util-scripts.md
+++ b/docs/contrib/util-scripts.md
@@ -32,7 +32,7 @@ This script offers a variety of utilities and helpers to enhance and simplify op
## brief descriptions of each function
-**Englist:**
+**English:**
1. `openim::util::ensure-gnu-sed` - Determines if GNU version of `sed` exists on the system and sets its name.
2. `openim::util::ensure-gnu-date` - Determines if GNU version of `date` exists on the system and sets its name.
3. `openim::util::check-file-in-alphabetical-order` - Checks if a file is sorted in alphabetical order.
diff --git a/docs/contributing/CONTRIBUTING-JP.md b/docs/contributing/CONTRIBUTING-JP.md
index 86bbfefcd..1798d4e3d 100644
--- a/docs/contributing/CONTRIBUTING-JP.md
+++ b/docs/contributing/CONTRIBUTING-JP.md
@@ -1,7 +1,7 @@
# How do I contribute code to OpenIM
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/contributing/CONTRIBUTING-PL.md b/docs/contributing/CONTRIBUTING-PL.md
index 86bbfefcd..1798d4e3d 100644
--- a/docs/contributing/CONTRIBUTING-PL.md
+++ b/docs/contributing/CONTRIBUTING-PL.md
@@ -1,7 +1,7 @@
# How do I contribute code to OpenIM
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_cs.md b/docs/readme/README_cs.md
index 5a9eeb232..63f730a51 100644
--- a/docs/readme/README_cs.md
+++ b/docs/readme/README_cs.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_da.md b/docs/readme/README_da.md
index 1b776ddb8..60d97348a 100644
--- a/docs/readme/README_da.md
+++ b/docs/readme/README_da.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_el.md b/docs/readme/README_el.md
index 252521f35..da01fcb47 100644
--- a/docs/readme/README_el.md
+++ b/docs/readme/README_el.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_es.md b/docs/readme/README_es.md
index cd1b7290e..f123b85c3 100644
--- a/docs/readme/README_es.md
+++ b/docs/readme/README_es.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_fa.md b/docs/readme/README_fa.md
index 49f05cd4c..7a1512e84 100644
--- a/docs/readme/README_fa.md
+++ b/docs/readme/README_fa.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_fr.md b/docs/readme/README_fr.md
index e707fc59b..aaf7a9bd4 100644
--- a/docs/readme/README_fr.md
+++ b/docs/readme/README_fr.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_hu.md b/docs/readme/README_hu.md
index 57f006692..61013c334 100644
--- a/docs/readme/README_hu.md
+++ b/docs/readme/README_hu.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_ja.md b/docs/readme/README_ja.md
index bd94b1153..5a083c1bf 100644
--- a/docs/readme/README_ja.md
+++ b/docs/readme/README_ja.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_ko.md b/docs/readme/README_ko.md
index bd7a1aed3..ebcdd71ee 100644
--- a/docs/readme/README_ko.md
+++ b/docs/readme/README_ko.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_tr.md b/docs/readme/README_tr.md
index ca2a816db..3cf19f537 100644
--- a/docs/readme/README_tr.md
+++ b/docs/readme/README_tr.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_uk.md b/docs/readme/README_uk.md
index 30bc76730..81820590b 100644
--- a/docs/readme/README_uk.md
+++ b/docs/readme/README_uk.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
diff --git a/docs/readme/README_vi.md b/docs/readme/README_vi.md
index e500da6d2..a6ab39253 100644
--- a/docs/readme/README_vi.md
+++ b/docs/readme/README_vi.md
@@ -19,7 +19,7 @@
- Englist ·
+ English ·
中文 ·
Українська ·
Česky ·
From 88ad85b5853ddc89efb078ca6e1b8e82d0537abf Mon Sep 17 00:00:00 2001
From: chao <48119764+withchao@users.noreply.github.com>
Date: Wed, 24 Jul 2024 15:06:26 +0800
Subject: [PATCH 06/17] fix: batchGetMaxSeq bug (#2438)
* fix: GroupApplicationAcceptedNotification
* fix: GroupApplicationAcceptedNotification
* fix: NotificationUserInfoUpdate
* cicd: robot automated Change
* fix: component
* fix: getConversationInfo
* feat: cron task
* feat: cron task
* feat: cron task
* feat: cron task
* feat: cron task
* fix: minio config url recognition error
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* friend incr sync
* friend incr sync
* friend incr sync
* friend incr sync
* friend incr sync
* mage
* optimization version log
* optimization version log
* sync
* sync
* sync
* group sync
* sync option
* sync option
* refactor: replace `friend` package with `realtion`.
* refactor: update lastest commit to relation.
* sync option
* sync option
* sync option
* sync
* sync
* go.mod
* seq
* update: go mod
* refactor: change incremental to full
* feat: get full friend user ids
* feat: api and config
* seq
* group version
* merge
* seq
* seq
* seq
* fix: sort by id avoid unstable sort friends.
* group
* group
* group
* fix: sort by id avoid unstable sort friends.
* fix: sort by id avoid unstable sort friends.
* fix: sort by id avoid unstable sort friends.
* user version
* seq
* seq
* seq user
* user online
* implement minio expire delete.
* user online
* config
* fix
* fix
* implement minio expire delete logic.
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* feat: implement scheduled delete outdated object in minio.
* update gomake version
* update gomake version
* implement FindExpires pagination.
* remove unnesseary incr.
* fix uncorrect args call.
* online push
* online push
* online push
* resolving conflicts
* resolving conflicts
* test
* api prommetrics
* api prommetrics
* api prommetrics
* api prommetrics
* api prommetrics
* rpc prommetrics
* rpc prommetrics
* online status
* online status
* online status
* online status
* sub
* conversation version incremental
* merge seq
* merge online
* merge online
* merge online
* merge seq
* GetOwnerConversation
* fix: change incremental syncer router name.
* rockscache batch get
* rockscache seq batch get
* fix: GetMsgDocModelByIndex bug
* update go.mod
* update go.mod
* merge
* feat: prometheus
* feat: prometheus
* group member sort
* sub
* sub
* fix: seq conversion bug
* fix: redis pipe exec
* sort version
* sort version
* sort version
* remove old version online subscription
* remove old version online subscription
* version log index
* version log index
* batch push
* batch push
* seq void filling
* fix: batchGetMaxSeq
* fix: batchGetMaxSeq
---------
Co-authored-by: withchao
Co-authored-by: Monet Lee
Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com>
Co-authored-by: icey-yu <1186114839@qq.com>
---
.../storage/cache/redis/seq_conversation.go | 4 ++--
pkg/common/storage/database/mgo/msg.go | 23 +++++++++++++++++++
2 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/pkg/common/storage/cache/redis/seq_conversation.go b/pkg/common/storage/cache/redis/seq_conversation.go
index fb8a547df..7fe849193 100644
--- a/pkg/common/storage/cache/redis/seq_conversation.go
+++ b/pkg/common/storage/cache/redis/seq_conversation.go
@@ -77,8 +77,8 @@ func (s *seqConversationCacheRedis) batchGetMaxSeq(ctx context.Context, keys []s
return errs.Wrap(err)
}
}
- if len(notFoundKey) > 0 {
- conversationID := keyConversationID[notFoundKey[0]]
+ for _, key := range notFoundKey {
+ conversationID := keyConversationID[key]
seq, err := s.GetMaxSeq(ctx, conversationID)
if err != nil {
return err
diff --git a/pkg/common/storage/database/mgo/msg.go b/pkg/common/storage/database/mgo/msg.go
index ebabadf52..2d1819c3a 100644
--- a/pkg/common/storage/database/mgo/msg.go
+++ b/pkg/common/storage/database/mgo/msg.go
@@ -94,6 +94,29 @@ func (m *MsgMgo) FindOneByDocID(ctx context.Context, docID string) (*model.MsgDo
}
func (m *MsgMgo) GetMsgBySeqIndexIn1Doc(ctx context.Context, userID, docID string, seqs []int64) ([]*model.MsgInfoModel, error) {
+ msgs, err := m.getMsgBySeqIndexIn1Doc(ctx, userID, docID, seqs)
+ if err != nil {
+ return nil, err
+ }
+ if len(msgs) == len(seqs) {
+ return msgs, nil
+ }
+ tmp := make(map[int64]*model.MsgInfoModel)
+ for i, val := range msgs {
+ tmp[val.Msg.Seq] = msgs[i]
+ }
+ res := make([]*model.MsgInfoModel, 0, len(seqs))
+ for _, seq := range seqs {
+ if val, ok := tmp[seq]; ok {
+ res = append(res, val)
+ } else {
+ res = append(res, &model.MsgInfoModel{Msg: &model.MsgDataModel{Seq: seq}})
+ }
+ }
+ return res, nil
+}
+
+func (m *MsgMgo) getMsgBySeqIndexIn1Doc(ctx context.Context, userID, docID string, seqs []int64) ([]*model.MsgInfoModel, error) {
indexs := make([]int64, 0, len(seqs))
for _, seq := range seqs {
indexs = append(indexs, m.model.GetMsgIndex(seq))
From 80b332cb8227997595ff69da5a5db234f9fab8e0 Mon Sep 17 00:00:00 2001
From: Monet Lee
Date: Wed, 24 Jul 2024 19:27:02 +0800
Subject: [PATCH 07/17] feat: implement log isSimpilfy. (#2436)
* feat: implement log isSimpilfy.
* update go mod.
---
config/log.yml | 3 ++-
go.mod | 4 ++--
go.sum | 8 ++++----
pkg/common/cmd/root.go | 1 +
pkg/common/config/config.go | 6 ++++--
5 files changed, 13 insertions(+), 9 deletions(-)
diff --git a/config/log.yml b/config/log.yml
index 2194d8917..8620af611 100644
--- a/config/log.yml
+++ b/config/log.yml
@@ -10,4 +10,5 @@ remainLogLevel: 6
isStdout: false
# Whether to log in JSON format, default is acceptable
isJson: false
-
+# output simplify log when KeyAndValues's value len is bigger than 50 in rpc method log
+isSimplify: true
\ No newline at end of file
diff --git a/go.mod b/go.mod
index fa40effdd..fe3f8afbc 100644
--- a/go.mod
+++ b/go.mod
@@ -12,8 +12,8 @@ require (
github.com/gorilla/websocket v1.5.1
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
github.com/mitchellh/mapstructure v1.5.0
- github.com/openimsdk/protocol v0.0.69-alpha.38
- github.com/openimsdk/tools v0.0.49-alpha.52
+ github.com/openimsdk/protocol v0.0.69-alpha.41
+ github.com/openimsdk/tools v0.0.49-alpha.55
github.com/pkg/errors v0.9.1 // indirect
github.com/prometheus/client_golang v1.18.0
github.com/stretchr/testify v1.9.0
diff --git a/go.sum b/go.sum
index cc0f5f766..e59418cd8 100644
--- a/go.sum
+++ b/go.sum
@@ -319,10 +319,10 @@ github.com/onsi/gomega v1.25.0 h1:Vw7br2PCDYijJHSfBOWhov+8cAnUf8MfMaIOV323l6Y=
github.com/onsi/gomega v1.25.0/go.mod h1:r+zV744Re+DiYCIPRlYOTxn0YkOLcAnW8k1xXdMPGhM=
github.com/openimsdk/gomake v0.0.14-alpha.5 h1:VY9c5x515lTfmdhhPjMvR3BBRrRquAUCFsz7t7vbv7Y=
github.com/openimsdk/gomake v0.0.14-alpha.5/go.mod h1:PndCozNc2IsQIciyn9mvEblYWZwJmAI+06z94EY+csI=
-github.com/openimsdk/protocol v0.0.69-alpha.38 h1:kVZCHIXg/el8YJFoIBWhZu1sbbTUqmzgF4l0W3sUH24=
-github.com/openimsdk/protocol v0.0.69-alpha.38/go.mod h1:OZQA9FR55lseYoN2Ql1XAHYKHJGu7OMNkUbuekrKCM8=
-github.com/openimsdk/tools v0.0.49-alpha.52 h1:NwAAtBO4BV96qG6Z0P2btGEqn4AI2DFgaHvLMXNHal0=
-github.com/openimsdk/tools v0.0.49-alpha.52/go.mod h1:h1cYmfyaVtgFbKmb1Cfsl8XwUOMTt8ubVUQrdGtsUh4=
+github.com/openimsdk/protocol v0.0.69-alpha.41 h1:9hoQ6UHMBq+g58KXir90EpnnvwJ1bvDPixPSaODo4nY=
+github.com/openimsdk/protocol v0.0.69-alpha.41/go.mod h1:OZQA9FR55lseYoN2Ql1XAHYKHJGu7OMNkUbuekrKCM8=
+github.com/openimsdk/tools v0.0.49-alpha.55 h1:KPgC53oqiwZYssLKljhtXbWXifMlTj2SSQEusj4Uf4k=
+github.com/openimsdk/tools v0.0.49-alpha.55/go.mod h1:h1cYmfyaVtgFbKmb1Cfsl8XwUOMTt8ubVUQrdGtsUh4=
github.com/pelletier/go-toml/v2 v2.1.0 h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4=
github.com/pelletier/go-toml/v2 v2.1.0/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc=
github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
diff --git a/pkg/common/cmd/root.go b/pkg/common/cmd/root.go
index 08bb6d064..84e985697 100644
--- a/pkg/common/cmd/root.go
+++ b/pkg/common/cmd/root.go
@@ -139,6 +139,7 @@ func (r *RootCmd) initializeLogger(cmdOpts *CmdOpts) error {
r.log.RemainRotationCount,
r.log.RotationTime,
config.Version,
+ r.log.IsSimplify,
)
if err != nil {
return errs.Wrap(err)
diff --git a/pkg/common/config/config.go b/pkg/common/config/config.go
index f018c7b83..c6c672eb8 100644
--- a/pkg/common/config/config.go
+++ b/pkg/common/config/config.go
@@ -15,6 +15,9 @@
package config
import (
+ "strings"
+ "time"
+
"github.com/openimsdk/tools/db/mongoutil"
"github.com/openimsdk/tools/db/redisutil"
"github.com/openimsdk/tools/mq/kafka"
@@ -22,8 +25,6 @@ import (
"github.com/openimsdk/tools/s3/kodo"
"github.com/openimsdk/tools/s3/minio"
"github.com/openimsdk/tools/s3/oss"
- "strings"
- "time"
)
type CacheConfig struct {
@@ -48,6 +49,7 @@ type Log struct {
RemainLogLevel int `mapstructure:"remainLogLevel"`
IsStdout bool `mapstructure:"isStdout"`
IsJson bool `mapstructure:"isJson"`
+ IsSimplify bool `mapstructure:"isSimplify"`
WithStack bool `mapstructure:"withStack"`
}
From ebdc91a966c1cf028ad9cbadbe44fb72e961e7b3 Mon Sep 17 00:00:00 2001
From: chao <48119764+withchao@users.noreply.github.com>
Date: Thu, 25 Jul 2024 20:01:33 +0800
Subject: [PATCH 08/17] fix: user seq bug (#2442)
* fix: GroupApplicationAcceptedNotification
* fix: GroupApplicationAcceptedNotification
* fix: NotificationUserInfoUpdate
* cicd: robot automated Change
* fix: component
* fix: getConversationInfo
* feat: cron task
* feat: cron task
* feat: cron task
* feat: cron task
* feat: cron task
* fix: minio config url recognition error
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* friend incr sync
* friend incr sync
* friend incr sync
* friend incr sync
* friend incr sync
* mage
* optimization version log
* optimization version log
* sync
* sync
* sync
* group sync
* sync option
* sync option
* refactor: replace `friend` package with `realtion`.
* refactor: update lastest commit to relation.
* sync option
* sync option
* sync option
* sync
* sync
* go.mod
* seq
* update: go mod
* refactor: change incremental to full
* feat: get full friend user ids
* feat: api and config
* seq
* group version
* merge
* seq
* seq
* seq
* fix: sort by id avoid unstable sort friends.
* group
* group
* group
* fix: sort by id avoid unstable sort friends.
* fix: sort by id avoid unstable sort friends.
* fix: sort by id avoid unstable sort friends.
* user version
* seq
* seq
* seq user
* user online
* implement minio expire delete.
* user online
* config
* fix
* fix
* implement minio expire delete logic.
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* feat: implement scheduled delete outdated object in minio.
* update gomake version
* update gomake version
* implement FindExpires pagination.
* remove unnesseary incr.
* fix uncorrect args call.
* online push
* online push
* online push
* resolving conflicts
* resolving conflicts
* test
* api prommetrics
* api prommetrics
* api prommetrics
* api prommetrics
* api prommetrics
* rpc prommetrics
* rpc prommetrics
* online status
* online status
* online status
* online status
* sub
* conversation version incremental
* merge seq
* merge online
* merge online
* merge online
* merge seq
* GetOwnerConversation
* fix: change incremental syncer router name.
* rockscache batch get
* rockscache seq batch get
* fix: GetMsgDocModelByIndex bug
* update go.mod
* update go.mod
* merge
* feat: prometheus
* feat: prometheus
* group member sort
* sub
* sub
* fix: seq conversion bug
* fix: redis pipe exec
* sort version
* sort version
* sort version
* remove old version online subscription
* remove old version online subscription
* version log index
* version log index
* batch push
* batch push
* seq void filling
* fix: batchGetMaxSeq
* fix: batchGetMaxSeq
* cache db error log
* 111
---------
Co-authored-by: withchao
Co-authored-by: Monet Lee
Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com>
Co-authored-by: icey-yu <1186114839@qq.com>
---
pkg/common/storage/cache/redis/batch.go | 2 +
.../storage/cache/redis/batch_handler.go | 1 +
pkg/common/storage/cache/redis/seq_user.go | 40 +++++++++----------
pkg/common/storage/cache/seq_user.go | 18 ++++-----
pkg/common/storage/controller/msg.go | 20 +++++-----
pkg/common/storage/database/mgo/seq_user.go | 14 +++----
pkg/common/storage/database/seq_user.go | 14 +++----
tools/seq/internal/main.go | 8 ++--
8 files changed, 60 insertions(+), 57 deletions(-)
diff --git a/pkg/common/storage/cache/redis/batch.go b/pkg/common/storage/cache/redis/batch.go
index 5f9a8c82d..4d65c5929 100644
--- a/pkg/common/storage/cache/redis/batch.go
+++ b/pkg/common/storage/cache/redis/batch.go
@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"github.com/dtm-labs/rockscache"
+ "github.com/openimsdk/tools/log"
"github.com/redis/go-redis/v9"
"golang.org/x/sync/singleflight"
"time"
@@ -49,6 +50,7 @@ func batchGetCache2[K comparable, V any](ctx context.Context, rcClient *rockscac
}
values, err := fn(ctx, queryIds)
if err != nil {
+ log.ZError(ctx, "batchGetCache query database failed", err, "keys", keys, "queryIds", queryIds)
return nil, err
}
if len(values) == 0 {
diff --git a/pkg/common/storage/cache/redis/batch_handler.go b/pkg/common/storage/cache/redis/batch_handler.go
index 52e046a40..f9923e198 100644
--- a/pkg/common/storage/cache/redis/batch_handler.go
+++ b/pkg/common/storage/cache/redis/batch_handler.go
@@ -118,6 +118,7 @@ func getCache[T any](ctx context.Context, rcClient *rockscache.Client, key strin
v, err := rcClient.Fetch2(ctx, key, expire, func() (s string, err error) {
t, err = fn(ctx)
if err != nil {
+ log.ZError(ctx, "getCache query database failed", err, "key", key)
return "", err
}
bs, err := json.Marshal(t)
diff --git a/pkg/common/storage/cache/redis/seq_user.go b/pkg/common/storage/cache/redis/seq_user.go
index 2ad43eebd..edbc66b21 100644
--- a/pkg/common/storage/cache/redis/seq_user.go
+++ b/pkg/common/storage/cache/redis/seq_user.go
@@ -44,38 +44,38 @@ func (s *seqUserCacheRedis) getSeqUserReadSeqKey(conversationID string, userID s
return cachekey.GetSeqUserReadSeqKey(conversationID, userID)
}
-func (s *seqUserCacheRedis) GetMaxSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
+func (s *seqUserCacheRedis) GetUserMaxSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
return getCache(ctx, s.rocks, s.getSeqUserMaxSeqKey(conversationID, userID), s.expireTime, func(ctx context.Context) (int64, error) {
- return s.mgo.GetMaxSeq(ctx, conversationID, userID)
+ return s.mgo.GetUserMaxSeq(ctx, conversationID, userID)
})
}
-func (s *seqUserCacheRedis) SetMaxSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
- if err := s.mgo.SetMaxSeq(ctx, conversationID, userID, seq); err != nil {
+func (s *seqUserCacheRedis) SetUserMaxSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
+ if err := s.mgo.SetUserMaxSeq(ctx, conversationID, userID, seq); err != nil {
return err
}
return s.rocks.TagAsDeleted2(ctx, s.getSeqUserMaxSeqKey(conversationID, userID))
}
-func (s *seqUserCacheRedis) GetMinSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
+func (s *seqUserCacheRedis) GetUserMinSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
return getCache(ctx, s.rocks, s.getSeqUserMinSeqKey(conversationID, userID), s.expireTime, func(ctx context.Context) (int64, error) {
- return s.mgo.GetMaxSeq(ctx, conversationID, userID)
+ return s.mgo.GetUserMinSeq(ctx, conversationID, userID)
})
}
-func (s *seqUserCacheRedis) SetMinSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
- return s.SetMinSeqs(ctx, userID, map[string]int64{conversationID: seq})
+func (s *seqUserCacheRedis) SetUserMinSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
+ return s.SetUserMinSeqs(ctx, userID, map[string]int64{conversationID: seq})
}
-func (s *seqUserCacheRedis) GetReadSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
+func (s *seqUserCacheRedis) GetUserReadSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
return getCache(ctx, s.rocks, s.getSeqUserReadSeqKey(conversationID, userID), s.readExpireTime, func(ctx context.Context) (int64, error) {
- return s.mgo.GetMaxSeq(ctx, conversationID, userID)
+ return s.mgo.GetUserReadSeq(ctx, conversationID, userID)
})
}
-func (s *seqUserCacheRedis) SetReadSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
+func (s *seqUserCacheRedis) SetUserReadSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
if seq%s.readSeqWriteRatio == 0 {
- if err := s.mgo.SetReadSeq(ctx, conversationID, userID, seq); err != nil {
+ if err := s.mgo.SetUserReadSeq(ctx, conversationID, userID, seq); err != nil {
return err
}
}
@@ -85,10 +85,10 @@ func (s *seqUserCacheRedis) SetReadSeq(ctx context.Context, conversationID strin
return nil
}
-func (s *seqUserCacheRedis) SetMinSeqs(ctx context.Context, userID string, seqs map[string]int64) error {
+func (s *seqUserCacheRedis) SetUserMinSeqs(ctx context.Context, userID string, seqs map[string]int64) error {
keys := make([]string, 0, len(seqs))
for conversationID, seq := range seqs {
- if err := s.mgo.SetMinSeq(ctx, conversationID, userID, seq); err != nil {
+ if err := s.mgo.SetUserMinSeq(ctx, conversationID, userID, seq); err != nil {
return err
}
keys = append(keys, s.getSeqUserMinSeqKey(conversationID, userID))
@@ -96,7 +96,7 @@ func (s *seqUserCacheRedis) SetMinSeqs(ctx context.Context, userID string, seqs
return DeleteCacheBySlot(ctx, s.rocks, keys)
}
-func (s *seqUserCacheRedis) setRedisReadSeqs(ctx context.Context, userID string, seqs map[string]int64) error {
+func (s *seqUserCacheRedis) setUserRedisReadSeqs(ctx context.Context, userID string, seqs map[string]int64) error {
keys := make([]string, 0, len(seqs))
keySeq := make(map[string]int64)
for conversationID, seq := range seqs {
@@ -121,16 +121,16 @@ func (s *seqUserCacheRedis) setRedisReadSeqs(ctx context.Context, userID string,
return nil
}
-func (s *seqUserCacheRedis) SetReadSeqs(ctx context.Context, userID string, seqs map[string]int64) error {
+func (s *seqUserCacheRedis) SetUserReadSeqs(ctx context.Context, userID string, seqs map[string]int64) error {
if len(seqs) == 0 {
return nil
}
- if err := s.setRedisReadSeqs(ctx, userID, seqs); err != nil {
+ if err := s.setUserRedisReadSeqs(ctx, userID, seqs); err != nil {
return err
}
for conversationID, seq := range seqs {
if seq%s.readSeqWriteRatio == 0 {
- if err := s.mgo.SetReadSeq(ctx, conversationID, userID, seq); err != nil {
+ if err := s.mgo.SetUserReadSeq(ctx, conversationID, userID, seq); err != nil {
return err
}
}
@@ -138,13 +138,13 @@ func (s *seqUserCacheRedis) SetReadSeqs(ctx context.Context, userID string, seqs
return nil
}
-func (s *seqUserCacheRedis) GetReadSeqs(ctx context.Context, userID string, conversationIDs []string) (map[string]int64, error) {
+func (s *seqUserCacheRedis) GetUserReadSeqs(ctx context.Context, userID string, conversationIDs []string) (map[string]int64, error) {
res, err := batchGetCache2(ctx, s.rocks, s.readExpireTime, conversationIDs, func(conversationID string) string {
return s.getSeqUserReadSeqKey(conversationID, userID)
}, func(v *readSeqModel) string {
return v.ConversationID
}, func(ctx context.Context, conversationIDs []string) ([]*readSeqModel, error) {
- seqs, err := s.mgo.GetReadSeqs(ctx, userID, conversationIDs)
+ seqs, err := s.mgo.GetUserReadSeqs(ctx, userID, conversationIDs)
if err != nil {
return nil, err
}
diff --git a/pkg/common/storage/cache/seq_user.go b/pkg/common/storage/cache/seq_user.go
index 4d0bb4ffa..61dbc0ab4 100644
--- a/pkg/common/storage/cache/seq_user.go
+++ b/pkg/common/storage/cache/seq_user.go
@@ -3,13 +3,13 @@ package cache
import "context"
type SeqUser interface {
- GetMaxSeq(ctx context.Context, conversationID string, userID string) (int64, error)
- SetMaxSeq(ctx context.Context, conversationID string, userID string, seq int64) error
- GetMinSeq(ctx context.Context, conversationID string, userID string) (int64, error)
- SetMinSeq(ctx context.Context, conversationID string, userID string, seq int64) error
- GetReadSeq(ctx context.Context, conversationID string, userID string) (int64, error)
- SetReadSeq(ctx context.Context, conversationID string, userID string, seq int64) error
- SetMinSeqs(ctx context.Context, userID string, seqs map[string]int64) error
- SetReadSeqs(ctx context.Context, userID string, seqs map[string]int64) error
- GetReadSeqs(ctx context.Context, userID string, conversationIDs []string) (map[string]int64, error)
+ GetUserMaxSeq(ctx context.Context, conversationID string, userID string) (int64, error)
+ SetUserMaxSeq(ctx context.Context, conversationID string, userID string, seq int64) error
+ GetUserMinSeq(ctx context.Context, conversationID string, userID string) (int64, error)
+ SetUserMinSeq(ctx context.Context, conversationID string, userID string, seq int64) error
+ GetUserReadSeq(ctx context.Context, conversationID string, userID string) (int64, error)
+ SetUserReadSeq(ctx context.Context, conversationID string, userID string, seq int64) error
+ SetUserMinSeqs(ctx context.Context, userID string, seqs map[string]int64) error
+ SetUserReadSeqs(ctx context.Context, userID string, seqs map[string]int64) error
+ GetUserReadSeqs(ctx context.Context, userID string, conversationIDs []string) (map[string]int64, error)
}
diff --git a/pkg/common/storage/controller/msg.go b/pkg/common/storage/controller/msg.go
index 32202ac9e..a6f0dbbb0 100644
--- a/pkg/common/storage/controller/msg.go
+++ b/pkg/common/storage/controller/msg.go
@@ -334,7 +334,7 @@ func (db *commonMsgDatabase) DeleteMessagesFromCache(ctx context.Context, conver
func (db *commonMsgDatabase) setHasReadSeqs(ctx context.Context, conversationID string, userSeqMap map[string]int64) error {
for userID, seq := range userSeqMap {
- if err := db.seqUser.SetReadSeq(ctx, conversationID, userID, seq); err != nil {
+ if err := db.seqUser.SetUserReadSeq(ctx, conversationID, userID, seq); err != nil {
return err
}
}
@@ -498,7 +498,7 @@ func (db *commonMsgDatabase) getMsgBySeqsRange(ctx context.Context, userID strin
// "userMinSeq" can be set as the same value as the conversation's "maxSeq" at the moment they join the group.
// This ensures that their message retrieval starts from the point they joined.
func (db *commonMsgDatabase) GetMsgBySeqsRange(ctx context.Context, userID string, conversationID string, begin, end, num, userMaxSeq int64) (int64, int64, []*sdkws.MsgData, error) {
- userMinSeq, err := db.seqUser.GetMinSeq(ctx, conversationID, userID)
+ userMinSeq, err := db.seqUser.GetUserMinSeq(ctx, conversationID, userID)
if err != nil && errs.Unwrap(err) != redis.Nil {
return 0, 0, nil, err
}
@@ -576,7 +576,7 @@ func (db *commonMsgDatabase) GetMsgBySeqsRange(ctx context.Context, userID strin
}
func (db *commonMsgDatabase) GetMsgBySeqs(ctx context.Context, userID string, conversationID string, seqs []int64) (int64, int64, []*sdkws.MsgData, error) {
- userMinSeq, err := db.seqUser.GetMinSeq(ctx, conversationID, userID)
+ userMinSeq, err := db.seqUser.GetUserMinSeq(ctx, conversationID, userID)
if err != nil && errs.Unwrap(err) != redis.Nil {
return 0, 0, nil, err
}
@@ -674,12 +674,12 @@ func (db *commonMsgDatabase) UserMsgsDestruct(ctx context.Context, userID string
log.ZDebug(ctx, "UserMsgsDestruct", "conversationID", conversationID, "userID", userID, "seqs", seqs)
if len(seqs) > 0 {
userMinSeq := seqs[len(seqs)-1] + 1
- currentUserMinSeq, err := db.seqUser.GetMinSeq(ctx, conversationID, userID)
+ currentUserMinSeq, err := db.seqUser.GetUserMinSeq(ctx, conversationID, userID)
if err != nil {
return nil, err
}
if currentUserMinSeq < userMinSeq {
- if err := db.seqUser.SetMinSeq(ctx, conversationID, userID, userMinSeq); err != nil {
+ if err := db.seqUser.SetUserMinSeq(ctx, conversationID, userID, userMinSeq); err != nil {
return nil, err
}
}
@@ -794,23 +794,23 @@ func (db *commonMsgDatabase) SetMinSeqs(ctx context.Context, seqs map[string]int
}
func (db *commonMsgDatabase) SetUserConversationsMinSeqs(ctx context.Context, userID string, seqs map[string]int64) error {
- return db.seqUser.SetMinSeqs(ctx, userID, seqs)
+ return db.seqUser.SetUserMinSeqs(ctx, userID, seqs)
}
func (db *commonMsgDatabase) UserSetHasReadSeqs(ctx context.Context, userID string, hasReadSeqs map[string]int64) error {
- return db.seqUser.SetReadSeqs(ctx, userID, hasReadSeqs)
+ return db.seqUser.SetUserReadSeqs(ctx, userID, hasReadSeqs)
}
func (db *commonMsgDatabase) SetHasReadSeq(ctx context.Context, userID string, conversationID string, hasReadSeq int64) error {
- return db.seqUser.SetReadSeq(ctx, conversationID, userID, hasReadSeq)
+ return db.seqUser.SetUserReadSeq(ctx, conversationID, userID, hasReadSeq)
}
func (db *commonMsgDatabase) GetHasReadSeqs(ctx context.Context, userID string, conversationIDs []string) (map[string]int64, error) {
- return db.seqUser.GetReadSeqs(ctx, userID, conversationIDs)
+ return db.seqUser.GetUserReadSeqs(ctx, userID, conversationIDs)
}
func (db *commonMsgDatabase) GetHasReadSeq(ctx context.Context, userID string, conversationID string) (int64, error) {
- return db.seqUser.GetReadSeq(ctx, conversationID, userID)
+ return db.seqUser.GetUserReadSeq(ctx, conversationID, userID)
}
func (db *commonMsgDatabase) SetSendMsgStatus(ctx context.Context, id string, status int32) error {
diff --git a/pkg/common/storage/database/mgo/seq_user.go b/pkg/common/storage/database/mgo/seq_user.go
index e0cbb08d9..7c9a8f133 100644
--- a/pkg/common/storage/database/mgo/seq_user.go
+++ b/pkg/common/storage/database/mgo/seq_user.go
@@ -68,27 +68,27 @@ func (s *seqUserMongo) getSeq(ctx context.Context, conversationID string, userID
}
}
-func (s *seqUserMongo) GetMaxSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
+func (s *seqUserMongo) GetUserMaxSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
return s.getSeq(ctx, conversationID, userID, "max_seq")
}
-func (s *seqUserMongo) SetMaxSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
+func (s *seqUserMongo) SetUserMaxSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
return s.setSeq(ctx, conversationID, userID, seq, "max_seq")
}
-func (s *seqUserMongo) GetMinSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
+func (s *seqUserMongo) GetUserMinSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
return s.getSeq(ctx, conversationID, userID, "min_seq")
}
-func (s *seqUserMongo) SetMinSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
+func (s *seqUserMongo) SetUserMinSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
return s.setSeq(ctx, conversationID, userID, seq, "min_seq")
}
-func (s *seqUserMongo) GetReadSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
+func (s *seqUserMongo) GetUserReadSeq(ctx context.Context, conversationID string, userID string) (int64, error) {
return s.getSeq(ctx, conversationID, userID, "read_seq")
}
-func (s *seqUserMongo) GetReadSeqs(ctx context.Context, userID string, conversationID []string) (map[string]int64, error) {
+func (s *seqUserMongo) GetUserReadSeqs(ctx context.Context, userID string, conversationID []string) (map[string]int64, error) {
if len(conversationID) == 0 {
return map[string]int64{}, nil
}
@@ -105,6 +105,6 @@ func (s *seqUserMongo) GetReadSeqs(ctx context.Context, userID string, conversat
return res, nil
}
-func (s *seqUserMongo) SetReadSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
+func (s *seqUserMongo) SetUserReadSeq(ctx context.Context, conversationID string, userID string, seq int64) error {
return s.setSeq(ctx, conversationID, userID, seq, "read_seq")
}
diff --git a/pkg/common/storage/database/seq_user.go b/pkg/common/storage/database/seq_user.go
index edd3910d0..9f75c710b 100644
--- a/pkg/common/storage/database/seq_user.go
+++ b/pkg/common/storage/database/seq_user.go
@@ -3,11 +3,11 @@ package database
import "context"
type SeqUser interface {
- GetMaxSeq(ctx context.Context, conversationID string, userID string) (int64, error)
- SetMaxSeq(ctx context.Context, conversationID string, userID string, seq int64) error
- GetMinSeq(ctx context.Context, conversationID string, userID string) (int64, error)
- SetMinSeq(ctx context.Context, conversationID string, userID string, seq int64) error
- GetReadSeq(ctx context.Context, conversationID string, userID string) (int64, error)
- SetReadSeq(ctx context.Context, conversationID string, userID string, seq int64) error
- GetReadSeqs(ctx context.Context, userID string, conversationID []string) (map[string]int64, error)
+ GetUserMaxSeq(ctx context.Context, conversationID string, userID string) (int64, error)
+ SetUserMaxSeq(ctx context.Context, conversationID string, userID string, seq int64) error
+ GetUserMinSeq(ctx context.Context, conversationID string, userID string) (int64, error)
+ SetUserMinSeq(ctx context.Context, conversationID string, userID string, seq int64) error
+ GetUserReadSeq(ctx context.Context, conversationID string, userID string) (int64, error)
+ SetUserReadSeq(ctx context.Context, conversationID string, userID string, seq int64) error
+ GetUserReadSeqs(ctx context.Context, userID string, conversationID []string) (map[string]int64, error)
}
diff --git a/tools/seq/internal/main.go b/tools/seq/internal/main.go
index 7f021a90e..2bec5a8f1 100644
--- a/tools/seq/internal/main.go
+++ b/tools/seq/internal/main.go
@@ -130,14 +130,14 @@ func Main(conf string, del time.Duration) error {
if err != nil {
return 0, err
}
- return uSeq.GetReadSeq(ctx, conversationID, userID)
+ return uSeq.GetUserReadSeq(ctx, conversationID, userID)
},
SetSeq: func(ctx context.Context, id string, seq int64) error {
conversationID, userID, err := uSpitHasReadSeq(id)
if err != nil {
return err
}
- return uSeq.SetReadSeq(ctx, conversationID, userID, seq)
+ return uSeq.SetUserReadSeq(ctx, conversationID, userID, seq)
},
},
{
@@ -147,14 +147,14 @@ func Main(conf string, del time.Duration) error {
if err != nil {
return 0, err
}
- return uSeq.GetMinSeq(ctx, conversationID, userID)
+ return uSeq.GetUserMinSeq(ctx, conversationID, userID)
},
SetSeq: func(ctx context.Context, id string, seq int64) error {
conversationID, userID, err := uSpitConversationUserMinSeq(id)
if err != nil {
return err
}
- return uSeq.SetMinSeq(ctx, conversationID, userID, seq)
+ return uSeq.SetUserMinSeq(ctx, conversationID, userID, seq)
},
},
}
From a8b84911a42bab22d0b38eb0cf47ba6606f0e98f Mon Sep 17 00:00:00 2001
From: icey-yu <119291641+icey-yu@users.noreply.github.com>
Date: Fri, 26 Jul 2024 14:40:36 +0800
Subject: [PATCH 09/17] fix: display is read (#2444)
---
internal/rpc/group/notification.go | 45 ++----------------------------
1 file changed, 3 insertions(+), 42 deletions(-)
diff --git a/internal/rpc/group/notification.go b/internal/rpc/group/notification.go
index a7398795f..9815167e9 100644
--- a/internal/rpc/group/notification.go
+++ b/internal/rpc/group/notification.go
@@ -17,6 +17,7 @@ package group
import (
"context"
"fmt"
+ "github.com/openimsdk/open-im-server/v3/pkg/common/convert"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/database"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/model"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/versionctx"
@@ -126,25 +127,8 @@ func (g *GroupNotificationSender) getGroupInfo(ctx context.Context, groupID stri
if len(ownerUserIDs) > 0 {
ownerUserID = ownerUserIDs[0]
}
- return &sdkws.GroupInfo{
- GroupID: gm.GroupID,
- GroupName: gm.GroupName,
- Notification: gm.Notification,
- Introduction: gm.Introduction,
- FaceURL: gm.FaceURL,
- OwnerUserID: ownerUserID,
- CreateTime: gm.CreateTime.UnixMilli(),
- MemberCount: num,
- Ex: gm.Ex,
- Status: gm.Status,
- CreatorUserID: gm.CreatorUserID,
- GroupType: gm.GroupType,
- NeedVerification: gm.NeedVerification,
- LookMemberInfo: gm.LookMemberInfo,
- ApplyMemberFriend: gm.ApplyMemberFriend,
- NotificationUpdateTime: gm.NotificationUpdateTime.UnixMilli(),
- NotificationUserID: gm.NotificationUserID,
- }, nil
+
+ return convert.Db2PbGroupInfo(gm, ownerUserID, num), nil
}
func (g *GroupNotificationSender) getGroupMembers(ctx context.Context, groupID string, userIDs []string) ([]*sdkws.GroupMemberFullInfo, error) {
@@ -198,29 +182,6 @@ func (g *GroupNotificationSender) getGroupOwnerAndAdminUserID(ctx context.Contex
return datautil.Slice(members, fn), nil
}
-//nolint:unused
-func (g *GroupNotificationSender) groupDB2PB(group *model.Group, ownerUserID string, memberCount uint32) *sdkws.GroupInfo {
- return &sdkws.GroupInfo{
- GroupID: group.GroupID,
- GroupName: group.GroupName,
- Notification: group.Notification,
- Introduction: group.Introduction,
- FaceURL: group.FaceURL,
- OwnerUserID: ownerUserID,
- CreateTime: group.CreateTime.UnixMilli(),
- MemberCount: memberCount,
- Ex: group.Ex,
- Status: group.Status,
- CreatorUserID: group.CreatorUserID,
- GroupType: group.GroupType,
- NeedVerification: group.NeedVerification,
- LookMemberInfo: group.LookMemberInfo,
- ApplyMemberFriend: group.ApplyMemberFriend,
- NotificationUpdateTime: group.NotificationUpdateTime.UnixMilli(),
- NotificationUserID: group.NotificationUserID,
- }
-}
-
func (g *GroupNotificationSender) groupMemberDB2PB(member *model.GroupMember, appMangerLevel int32) *sdkws.GroupMemberFullInfo {
return &sdkws.GroupMemberFullInfo{
GroupID: member.GroupID,
From c3c9969f2f15f182f280f303aa5cc870ca56bb99 Mon Sep 17 00:00:00 2001
From: Monet Lee
Date: Fri, 26 Jul 2024 15:52:37 +0800
Subject: [PATCH 10/17] chore: add debug log in writePongMsg (#2446)
* update protocol in go mod.
* add debug log in writePongMsg.
* update log level.
---
go.mod | 2 +-
go.sum | 4 ++--
internal/msggateway/client.go | 12 +++++++++---
3 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/go.mod b/go.mod
index fe3f8afbc..71301d290 100644
--- a/go.mod
+++ b/go.mod
@@ -12,7 +12,7 @@ require (
github.com/gorilla/websocket v1.5.1
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
github.com/mitchellh/mapstructure v1.5.0
- github.com/openimsdk/protocol v0.0.69-alpha.41
+ github.com/openimsdk/protocol v0.0.69-alpha.42
github.com/openimsdk/tools v0.0.49-alpha.55
github.com/pkg/errors v0.9.1 // indirect
github.com/prometheus/client_golang v1.18.0
diff --git a/go.sum b/go.sum
index e59418cd8..53060b198 100644
--- a/go.sum
+++ b/go.sum
@@ -319,8 +319,8 @@ github.com/onsi/gomega v1.25.0 h1:Vw7br2PCDYijJHSfBOWhov+8cAnUf8MfMaIOV323l6Y=
github.com/onsi/gomega v1.25.0/go.mod h1:r+zV744Re+DiYCIPRlYOTxn0YkOLcAnW8k1xXdMPGhM=
github.com/openimsdk/gomake v0.0.14-alpha.5 h1:VY9c5x515lTfmdhhPjMvR3BBRrRquAUCFsz7t7vbv7Y=
github.com/openimsdk/gomake v0.0.14-alpha.5/go.mod h1:PndCozNc2IsQIciyn9mvEblYWZwJmAI+06z94EY+csI=
-github.com/openimsdk/protocol v0.0.69-alpha.41 h1:9hoQ6UHMBq+g58KXir90EpnnvwJ1bvDPixPSaODo4nY=
-github.com/openimsdk/protocol v0.0.69-alpha.41/go.mod h1:OZQA9FR55lseYoN2Ql1XAHYKHJGu7OMNkUbuekrKCM8=
+github.com/openimsdk/protocol v0.0.69-alpha.42 h1:Vwuru2NtyTHuqaM+1JGxcoGvP25QWjS92oI0zGJp+lM=
+github.com/openimsdk/protocol v0.0.69-alpha.42/go.mod h1:OZQA9FR55lseYoN2Ql1XAHYKHJGu7OMNkUbuekrKCM8=
github.com/openimsdk/tools v0.0.49-alpha.55 h1:KPgC53oqiwZYssLKljhtXbWXifMlTj2SSQEusj4Uf4k=
github.com/openimsdk/tools v0.0.49-alpha.55/go.mod h1:h1cYmfyaVtgFbKmb1Cfsl8XwUOMTt8ubVUQrdGtsUh4=
github.com/pelletier/go-toml/v2 v2.1.0 h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4=
diff --git a/internal/msggateway/client.go b/internal/msggateway/client.go
index 889e5c456..ded830c43 100644
--- a/internal/msggateway/client.go
+++ b/internal/msggateway/client.go
@@ -107,7 +107,6 @@ func (c *Client) pingHandler(appData string) error {
}
log.ZDebug(c.ctx, "ping Handler Success.", "appData", appData)
-
return c.writePongMsg(appData)
}
@@ -392,17 +391,24 @@ func (c *Client) writePingMsg() error {
}
func (c *Client) writePongMsg(appData string) error {
+ log.ZDebug(c.ctx, "write Pong Msg in Server", "appData", appData)
if c.closed.Load() {
return nil
}
+ log.ZDebug(c.ctx, "write Pong Msg in Server", "appData", appData)
c.w.Lock()
defer c.w.Unlock()
+ log.ZDebug(c.ctx, "write Pong Msg in Server", "appData", appData)
err := c.conn.SetWriteDeadline(writeWait)
if err != nil {
- return err
+ return errs.Wrap(err)
+ }
+ err = c.conn.WriteMessage(PongMessage, []byte(appData))
+ if err != nil {
+ log.ZWarn(c.ctx, "Write Message have error", errs.Wrap(err), "Pong msg", PongMessage)
}
- return c.conn.WriteMessage(PongMessage, []byte(appData))
+ return errs.Wrap(err)
}
From 5c52840d670e1a280335015d648bc090577cb068 Mon Sep 17 00:00:00 2001
From: chao <48119764+withchao@users.noreply.github.com>
Date: Fri, 26 Jul 2024 15:53:25 +0800
Subject: [PATCH 11/17] fix: user seq, asynchronous friend notification,
message search (#2447)
* fix: GroupApplicationAcceptedNotification
* fix: GroupApplicationAcceptedNotification
* fix: NotificationUserInfoUpdate
* cicd: robot automated Change
* fix: component
* fix: getConversationInfo
* feat: cron task
* feat: cron task
* feat: cron task
* feat: cron task
* feat: cron task
* fix: minio config url recognition error
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* new mongo
* friend incr sync
* friend incr sync
* friend incr sync
* friend incr sync
* friend incr sync
* mage
* optimization version log
* optimization version log
* sync
* sync
* sync
* group sync
* sync option
* sync option
* refactor: replace `friend` package with `realtion`.
* refactor: update lastest commit to relation.
* sync option
* sync option
* sync option
* sync
* sync
* go.mod
* seq
* update: go mod
* refactor: change incremental to full
* feat: get full friend user ids
* feat: api and config
* seq
* group version
* merge
* seq
* seq
* seq
* fix: sort by id avoid unstable sort friends.
* group
* group
* group
* fix: sort by id avoid unstable sort friends.
* fix: sort by id avoid unstable sort friends.
* fix: sort by id avoid unstable sort friends.
* user version
* seq
* seq
* seq user
* user online
* implement minio expire delete.
* user online
* config
* fix
* fix
* implement minio expire delete logic.
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* online cache
* feat: implement scheduled delete outdated object in minio.
* update gomake version
* update gomake version
* implement FindExpires pagination.
* remove unnesseary incr.
* fix uncorrect args call.
* online push
* online push
* online push
* resolving conflicts
* resolving conflicts
* test
* api prommetrics
* api prommetrics
* api prommetrics
* api prommetrics
* api prommetrics
* rpc prommetrics
* rpc prommetrics
* online status
* online status
* online status
* online status
* sub
* conversation version incremental
* merge seq
* merge online
* merge online
* merge online
* merge seq
* GetOwnerConversation
* fix: change incremental syncer router name.
* rockscache batch get
* rockscache seq batch get
* fix: GetMsgDocModelByIndex bug
* update go.mod
* update go.mod
* merge
* feat: prometheus
* feat: prometheus
* group member sort
* sub
* sub
* fix: seq conversion bug
* fix: redis pipe exec
* sort version
* sort version
* sort version
* remove old version online subscription
* remove old version online subscription
* version log index
* version log index
* batch push
* batch push
* seq void filling
* fix: batchGetMaxSeq
* fix: batchGetMaxSeq
* cache db error log
* 111
* fix bug
---------
Co-authored-by: withchao
Co-authored-by: Monet Lee
Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com>
Co-authored-by: icey-yu <1186114839@qq.com>
---
internal/rpc/friend/friend.go | 4 +-
internal/rpc/friend/sync.go | 22 +-
internal/rpc/msg/sync_msg.go | 4 +-
pkg/common/storage/cache/redis/batch_test.go | 2 +-
.../storage/cache/redis/seq_user_test.go | 32 ++
pkg/common/storage/controller/msg.go | 4 +-
pkg/common/storage/database/mgo/msg.go | 457 ++++++++++++++----
pkg/common/storage/database/mgo/msg_test.go | 75 +++
.../database/mgo/seq_conversation_test.go | 7 +-
pkg/common/storage/database/mgo/seq_user.go | 9 +
pkg/common/storage/database/msg.go | 2 +-
11 files changed, 518 insertions(+), 100 deletions(-)
create mode 100644 pkg/common/storage/database/mgo/msg_test.go
diff --git a/internal/rpc/friend/friend.go b/internal/rpc/friend/friend.go
index 622e19f42..bdb786bca 100644
--- a/internal/rpc/friend/friend.go
+++ b/internal/rpc/friend/friend.go
@@ -16,6 +16,7 @@ package friend
import (
"context"
+ "github.com/openimsdk/tools/mq/memamq"
"github.com/openimsdk/open-im-server/v3/pkg/common/config"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/cache/redis"
@@ -49,6 +50,7 @@ type friendServer struct {
RegisterCenter discovery.SvcDiscoveryRegistry
config *Config
webhookClient *webhook.Client
+ queue *memamq.MemoryQueue
}
type Config struct {
@@ -118,8 +120,8 @@ func Start(ctx context.Context, config *Config, client discovery.SvcDiscoveryReg
conversationRpcClient: rpcclient.NewConversationRpcClient(client, config.Share.RpcRegisterName.Conversation),
config: config,
webhookClient: webhook.NewWebhookClient(config.WebhooksConfig.URL),
+ queue: memamq.NewMemoryQueue(128, 1024*8),
})
-
return nil
}
diff --git a/internal/rpc/friend/sync.go b/internal/rpc/friend/sync.go
index 145c287da..902cc7303 100644
--- a/internal/rpc/friend/sync.go
+++ b/internal/rpc/friend/sync.go
@@ -4,6 +4,7 @@ import (
"context"
"github.com/openimsdk/open-im-server/v3/pkg/util/hashutil"
"github.com/openimsdk/protocol/sdkws"
+ "github.com/openimsdk/tools/log"
"slices"
"github.com/openimsdk/open-im-server/v3/internal/rpc/incrversion"
@@ -17,14 +18,23 @@ func (s *friendServer) NotificationUserInfoUpdate(ctx context.Context, req *rela
if err != nil {
return nil, err
}
- for _, userID := range userIDs {
- if err := s.db.OwnerIncrVersion(ctx, userID, []string{req.UserID}, model.VersionStateUpdate); err != nil {
- return nil, err
+ if len(userIDs) > 0 {
+ friendUserIDs := []string{req.UserID}
+ noCancelCtx := context.WithoutCancel(ctx)
+ err := s.queue.PushCtx(ctx, func() {
+ for _, userID := range userIDs {
+ if err := s.db.OwnerIncrVersion(noCancelCtx, userID, friendUserIDs, model.VersionStateUpdate); err != nil {
+ log.ZError(ctx, "OwnerIncrVersion", err, "userID", userID, "friendUserIDs", friendUserIDs)
+ }
+ }
+ for _, userID := range userIDs {
+ s.notificationSender.FriendInfoUpdatedNotification(noCancelCtx, req.UserID, userID)
+ }
+ })
+ if err != nil {
+ log.ZError(ctx, "NotificationUserInfoUpdate timeout", err, "userID", req.UserID)
}
}
- for _, userID := range userIDs {
- s.notificationSender.FriendInfoUpdatedNotification(ctx, req.UserID, userID)
- }
return &relation.NotificationUserInfoUpdateResp{}, nil
}
diff --git a/internal/rpc/msg/sync_msg.go b/internal/rpc/msg/sync_msg.go
index afb79506e..f5b5ebda5 100644
--- a/internal/rpc/msg/sync_msg.go
+++ b/internal/rpc/msg/sync_msg.go
@@ -111,7 +111,7 @@ func (m *msgServer) GetMaxSeq(ctx context.Context, req *sdkws.GetMaxSeqReq) (*sd
func (m *msgServer) SearchMessage(ctx context.Context, req *msg.SearchMessageReq) (resp *msg.SearchMessageResp, err error) {
var chatLogs []*sdkws.MsgData
- var total int32
+ var total int64
resp = &msg.SearchMessageResp{}
if total, chatLogs, err = m.MsgDatabase.SearchMessage(ctx, req); err != nil {
return nil, err
@@ -194,7 +194,7 @@ func (m *msgServer) SearchMessage(ctx context.Context, req *msg.SearchMessageReq
}
resp.ChatLogs = append(resp.ChatLogs, pbchatLog)
}
- resp.ChatLogsNum = total
+ resp.ChatLogsNum = int32(total)
return resp, nil
}
diff --git a/pkg/common/storage/cache/redis/batch_test.go b/pkg/common/storage/cache/redis/batch_test.go
index e4caa2a21..bbb6d76f1 100644
--- a/pkg/common/storage/cache/redis/batch_test.go
+++ b/pkg/common/storage/cache/redis/batch_test.go
@@ -45,7 +45,7 @@ func TestName(t *testing.T) {
}
seqUser := NewSeqUserCacheRedis(rdb, mgoSeqUser)
- res, err := seqUser.GetReadSeqs(ctx, "2110910952", []string{"sg_2920732023", "sg_345762580"})
+ res, err := seqUser.GetUserReadSeqs(ctx, "2110910952", []string{"sg_2920732023", "sg_345762580"})
if err != nil {
panic(err)
}
diff --git a/pkg/common/storage/cache/redis/seq_user_test.go b/pkg/common/storage/cache/redis/seq_user_test.go
index e4fd95922..0059c81db 100644
--- a/pkg/common/storage/cache/redis/seq_user_test.go
+++ b/pkg/common/storage/cache/redis/seq_user_test.go
@@ -4,7 +4,10 @@ import (
"context"
"fmt"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/cache/cachekey"
+ mgo2 "github.com/openimsdk/open-im-server/v3/pkg/common/storage/database/mgo"
"github.com/redis/go-redis/v9"
+ "go.mongodb.org/mongo-driver/mongo"
+ "go.mongodb.org/mongo-driver/mongo/options"
"log"
"strconv"
"sync/atomic"
@@ -77,3 +80,32 @@ func TestRecvOnline(t *testing.T) {
fmt.Printf("Received message from channel %s: %s\n", msg.Channel, msg.Payload)
}
}
+
+func TestName1(t *testing.T) {
+ opt := &redis.Options{
+ Addr: "172.16.8.48:16379",
+ Password: "openIM123",
+ DB: 0,
+ }
+ rdb := redis.NewClient(opt)
+
+ mgo, err := mongo.Connect(context.Background(),
+ options.Client().
+ ApplyURI("mongodb://openIM:openIM123@172.16.8.48:37017/openim_v3?maxPoolSize=100").
+ SetConnectTimeout(5*time.Second))
+ if err != nil {
+ panic(err)
+ }
+ model, err := mgo2.NewSeqUserMongo(mgo.Database("openim_v3"))
+ if err != nil {
+ panic(err)
+ }
+ seq := NewSeqUserCacheRedis(rdb, model)
+
+ res, err := seq.GetUserReadSeqs(context.Background(), "2110910952", []string{"sg_345762580", "2000", "3000"})
+ if err != nil {
+ panic(err)
+ }
+ t.Log(res)
+
+}
diff --git a/pkg/common/storage/controller/msg.go b/pkg/common/storage/controller/msg.go
index a6f0dbbb0..4ea74ef69 100644
--- a/pkg/common/storage/controller/msg.go
+++ b/pkg/common/storage/controller/msg.go
@@ -84,7 +84,7 @@ type CommonMsgDatabase interface {
//GetConversationMinMaxSeqInMongoAndCache(ctx context.Context, conversationID string) (minSeqMongo, maxSeqMongo, minSeqCache, maxSeqCache int64, err error)
SetSendMsgStatus(ctx context.Context, id string, status int32) error
GetSendMsgStatus(ctx context.Context, id string) (int32, error)
- SearchMessage(ctx context.Context, req *pbmsg.SearchMessageReq) (total int32, msgData []*sdkws.MsgData, err error)
+ SearchMessage(ctx context.Context, req *pbmsg.SearchMessageReq) (total int64, msgData []*sdkws.MsgData, err error)
FindOneByDocIDs(ctx context.Context, docIDs []string, seqs map[string]int64) (map[string]*sdkws.MsgData, error)
// to mq
@@ -878,7 +878,7 @@ func (db *commonMsgDatabase) RangeGroupSendCount(
return db.msgDocDatabase.RangeGroupSendCount(ctx, start, end, ase, pageNumber, showNumber)
}
-func (db *commonMsgDatabase) SearchMessage(ctx context.Context, req *pbmsg.SearchMessageReq) (total int32, msgData []*sdkws.MsgData, err error) {
+func (db *commonMsgDatabase) SearchMessage(ctx context.Context, req *pbmsg.SearchMessageReq) (total int64, msgData []*sdkws.MsgData, err error) {
var totalMsgs []*sdkws.MsgData
total, msgs, err := db.msgDocDatabase.SearchMessage(ctx, req)
if err != nil {
diff --git a/pkg/common/storage/database/mgo/msg.go b/pkg/common/storage/database/mgo/msg.go
index 2d1819c3a..03f47c503 100644
--- a/pkg/common/storage/database/mgo/msg.go
+++ b/pkg/common/storage/database/mgo/msg.go
@@ -278,124 +278,409 @@ func (m *MsgMgo) MarkSingleChatMsgsAsRead(ctx context.Context, userID string, do
return nil
}
-func (m *MsgMgo) SearchMessage(ctx context.Context, req *msg.SearchMessageReq) (int32, []*model.MsgInfoModel, error) {
- where := make(bson.A, 0, 6)
+//func (m *MsgMgo) searchCount(ctx context.Context, filter any) (int64, error) {
+//
+// return nil, nil
+//}
+
+//func (m *MsgMgo) searchMessage(ctx context.Context, filter any, nextID primitive.ObjectID, content bool, limit int) (int64, []*model.MsgInfoModel, primitive.ObjectID, error) {
+// var pipeline bson.A
+// if !nextID.IsZero() {
+// pipeline = append(pipeline, bson.M{"$match": bson.M{"_id": bson.M{"$gt": nextID}}})
+// }
+// pipeline = append(pipeline,
+// bson.M{"$match": filter},
+// bson.M{"$limit": limit},
+// bson.M{"$unwind": "$msgs"},
+// bson.M{"$match": filter},
+// bson.M{
+// "$group": bson.M{
+// "_id": "$_id",
+// "doc_id": bson.M{
+// "$first": "$doc_id",
+// },
+// "msgs": bson.M{"$push": "$msgs"},
+// },
+// },
+// )
+// if !content {
+// pipeline = append(pipeline,
+// bson.M{
+// "$project": bson.M{
+// "_id": 1,
+// "count": bson.M{"$size": "$msgs"},
+// },
+// },
+// )
+// type result struct {
+// ID primitive.ObjectID `bson:"_id"`
+// Count int64 `bson:"count"`
+// }
+// res, err := mongoutil.Aggregate[result](ctx, m.coll, pipeline)
+// if err != nil {
+// return 0, nil, primitive.ObjectID{}, err
+// }
+// if len(res) == 0 {
+// return 0, nil, primitive.ObjectID{}, nil
+// }
+// var count int64
+// for _, r := range res {
+// count += r.Count
+// }
+// return count, nil, res[len(res)-1].ID, nil
+// }
+// type result struct {
+// ID primitive.ObjectID `bson:"_id"`
+// Msg []*model.MsgInfoModel `bson:"msgs"`
+// }
+// res, err := mongoutil.Aggregate[result](ctx, m.coll, pipeline)
+// if err != nil {
+// return 0, nil, primitive.ObjectID{}, err
+// }
+// if len(res) == 0 {
+// return 0, nil, primitive.ObjectID{}, err
+// }
+// var count int
+// for _, r := range res {
+// count += len(r.Msg)
+// }
+// msgs := make([]*model.MsgInfoModel, 0, count)
+// for _, r := range res {
+// msgs = append(msgs, r.Msg...)
+// }
+// return int64(count), msgs, res[len(res)-1].ID, nil
+//}
+
+/*
+
+db.msg3.aggregate(
+ [
+ {
+ "$match": {
+ "doc_id": "si_7009965934_8710838466:0"
+ },
+
+ }
+ ]
+)
+
+
+*/
+
+type searchMessageIndex struct {
+ ID primitive.ObjectID `bson:"_id"`
+ Index []int64 `bson:"index"`
+}
+
+func (m *MsgMgo) searchMessageIndex(ctx context.Context, filter any, nextID primitive.ObjectID, limit int) ([]searchMessageIndex, error) {
+ var pipeline bson.A
+ if !nextID.IsZero() {
+ pipeline = append(pipeline, bson.M{"$match": bson.M{"_id": bson.M{"$gt": nextID}}})
+ }
+ pipeline = append(pipeline,
+ bson.M{"$sort": bson.M{"_id": 1}},
+ bson.M{"$match": filter},
+ bson.M{"$limit": limit},
+ bson.M{
+ "$project": bson.M{
+ "_id": 1,
+ "msgs": bson.M{
+ "$map": bson.M{
+ "input": "$msgs",
+ "as": "msg",
+ "in": bson.M{
+ "$mergeObjects": bson.A{
+ "$$msg",
+ bson.M{
+ "_search_temp_index": bson.M{
+ "$indexOfArray": bson.A{
+ "$msgs", "$$msg",
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ bson.M{"$unwind": "$msgs"},
+ bson.M{"$match": filter},
+ bson.M{
+ "$project": bson.M{
+ "_id": 1,
+ "msgs._search_temp_index": 1,
+ },
+ },
+ bson.M{
+ "$group": bson.M{
+ "_id": "$_id",
+ "index": bson.M{"$push": "$msgs._search_temp_index"},
+ },
+ },
+ bson.M{"$sort": bson.M{"_id": 1}},
+ )
+ return mongoutil.Aggregate[searchMessageIndex](ctx, m.coll, pipeline)
+}
+
+func (m *MsgMgo) searchMessage(ctx context.Context, req *msg.SearchMessageReq) (int64, []searchMessageIndex, error) {
+ filter := bson.M{}
if req.RecvID != "" {
- where = append(where, bson.M{"msgs.msg.recv_id": req.RecvID})
+ filter["$or"] = bson.A{
+ bson.M{"msgs.msg.recv_id": req.RecvID},
+ bson.M{"msgs.msg.group_id": req.RecvID},
+ }
}
if req.SendID != "" {
- where = append(where, bson.M{"msgs.msg.send_id": req.SendID})
+ filter["msgs.msg.send_id"] = req.SendID
}
if req.ContentType != 0 {
- where = append(where, bson.M{"msgs.msg.content_type": req.ContentType})
+ filter["msgs.msg.content_type"] = req.ContentType
}
if req.SessionType != 0 {
- where = append(where, bson.M{"msgs.msg.session_type": req.SessionType})
+ filter["msgs.msg.session_type"] = req.SessionType
}
if req.SendTime != "" {
sendTime, err := time.Parse(time.DateOnly, req.SendTime)
if err != nil {
return 0, nil, errs.ErrArgs.WrapMsg("invalid sendTime", "req", req.SendTime, "format", time.DateOnly, "cause", err.Error())
}
- where = append(where,
- bson.M{
- "msgs.msg.send_time": bson.M{
- "$gte": sendTime.UnixMilli(),
- },
- },
+ filter["$and"] = bson.A{
+ bson.M{"msgs.msg.send_time": bson.M{
+ "$gte": sendTime.UnixMilli(),
+ }},
bson.M{
"msgs.msg.send_time": bson.M{
"$lt": sendTime.Add(time.Hour * 24).UnixMilli(),
},
},
- )
- }
- pipeline := bson.A{
- bson.M{
- "$unwind": "$msgs",
- },
- }
- if len(where) > 0 {
- pipeline = append(pipeline, bson.M{
- "$match": bson.M{"$and": where},
- })
+ }
}
- pipeline = append(pipeline,
- bson.M{
- "$project": bson.M{
- "_id": 0,
- "msg": "$msgs.msg",
- },
- },
- bson.M{
- "$count": "count",
- },
+
+ var (
+ nextID primitive.ObjectID
+ count int
+ dataRange []searchMessageIndex
+ skip = int((req.Pagination.GetPageNumber() - 1) * req.Pagination.GetShowNumber())
)
- count, err := mongoutil.Aggregate[int32](ctx, m.coll, pipeline)
- if err != nil {
- return 0, nil, err
+ _, _ = dataRange, skip
+ const maxDoc = 50
+ data := make([]searchMessageIndex, 0, req.Pagination.GetShowNumber())
+ push := cap(data)
+ for i := 0; ; i++ {
+ res, err := m.searchMessageIndex(ctx, filter, nextID, maxDoc)
+ if err != nil {
+ return 0, nil, err
+ }
+ if len(res) > 0 {
+ nextID = res[len(res)-1].ID
+ }
+ for _, r := range res {
+ var dataIndex []int64
+ for _, index := range r.Index {
+ if push > 0 && count >= skip {
+ dataIndex = append(dataIndex, index)
+ push--
+ }
+ count++
+ }
+ if len(dataIndex) > 0 {
+ data = append(data, searchMessageIndex{
+ ID: r.ID,
+ Index: dataIndex,
+ })
+ }
+ }
+ if push <= 0 {
+ push--
+ }
+ if len(res) < maxDoc || push < -10 {
+ return int64(count), data, nil
+ }
+ }
+}
+
+func (m *MsgMgo) getDocRange(ctx context.Context, id primitive.ObjectID, index []int64) ([]*model.MsgInfoModel, error) {
+ if len(index) == 0 {
+ return nil, nil
}
- if len(count) == 0 || count[0] == 0 {
- return 0, nil, nil
+
+ pipeline := bson.A{
+ bson.M{"$match": bson.M{"_id": id}},
+ bson.M{"$project": "$msgs"},
}
- pipeline = pipeline[:len(pipeline)-1]
- pipeline = append(pipeline,
- bson.M{
- "$skip": (req.Pagination.GetPageNumber() - 1) * req.Pagination.GetShowNumber(),
- },
- bson.M{
- "$limit": req.Pagination.GetShowNumber(),
- },
- )
msgs, err := mongoutil.Aggregate[*model.MsgInfoModel](ctx, m.coll, pipeline)
+ if err != nil {
+ return nil, err
+ }
+ return msgs, nil
+}
+
+func (m *MsgMgo) SearchMessage(ctx context.Context, req *msg.SearchMessageReq) (int64, []*model.MsgInfoModel, error) {
+ count, data, err := m.searchMessage(ctx, req)
if err != nil {
return 0, nil, err
}
- for i := range msgs {
- msgInfo := msgs[i]
- if msgInfo == nil || msgInfo.Msg == nil {
- continue
+ var msgs []*model.MsgInfoModel
+ if len(data) > 0 {
+ var n int
+ for _, d := range data {
+ n += len(d.Index)
}
- if msgInfo.Revoke != nil {
- revokeContent := sdkws.MessageRevokedContent{
- RevokerID: msgInfo.Revoke.UserID,
- RevokerRole: msgInfo.Revoke.Role,
- ClientMsgID: msgInfo.Msg.ClientMsgID,
- RevokerNickname: msgInfo.Revoke.Nickname,
- RevokeTime: msgInfo.Revoke.Time,
- SourceMessageSendTime: msgInfo.Msg.SendTime,
- SourceMessageSendID: msgInfo.Msg.SendID,
- SourceMessageSenderNickname: msgInfo.Msg.SenderNickname,
- SessionType: msgInfo.Msg.SessionType,
- Seq: msgInfo.Msg.Seq,
- Ex: msgInfo.Msg.Ex,
- }
- data, err := jsonutil.JsonMarshal(&revokeContent)
- if err != nil {
- return 0, nil, errs.WrapMsg(err, "json.Marshal revokeContent")
- }
- elem := sdkws.NotificationElem{Detail: string(data)}
- content, err := jsonutil.JsonMarshal(&elem)
- if err != nil {
- return 0, nil, errs.WrapMsg(err, "json.Marshal elem")
+ msgs = make([]*model.MsgInfoModel, 0, n)
+ }
+ for _, val := range data {
+ res, err := mongoutil.FindOne[*model.MsgDocModel](ctx, m.coll, bson.M{"_id": val.ID})
+ if err != nil {
+ return 0, nil, err
+ }
+ for _, i := range val.Index {
+ if i >= int64(len(res.Msg)) {
+ continue
}
- msgInfo.Msg.ContentType = constant.MsgRevokeNotification
- msgInfo.Msg.Content = string(content)
+ msgs = append(msgs, res.Msg[i])
}
}
- //start := (req.Pagination.PageNumber - 1) * req.Pagination.ShowNumber
- //n := int32(len(msgs))
- //if start >= n {
- // return n, []*relation.MsgInfoModel{}, nil
- //}
- //if start+req.Pagination.ShowNumber < n {
- // msgs = msgs[start : start+req.Pagination.ShowNumber]
- //} else {
- // msgs = msgs[start:]
- //}
- return count[0], msgs, nil
+ return count, msgs, nil
}
+//func (m *MsgMgo) SearchMessage(ctx context.Context, req *msg.SearchMessageReq) (int32, []*model.MsgInfoModel, error) {
+// where := make(bson.A, 0, 6)
+// if req.RecvID != "" {
+// if req.SessionType == constant.ReadGroupChatType {
+// where = append(where, bson.M{
+// "$or": bson.A{
+// bson.M{"doc_id": "^n_" + req.RecvID + ":"},
+// bson.M{"doc_id": "^sg_" + req.RecvID + ":"},
+// },
+// })
+// } else {
+// where = append(where, bson.M{"msgs.msg.recv_id": req.RecvID})
+// }
+// }
+// if req.SendID != "" {
+// where = append(where, bson.M{"msgs.msg.send_id": req.SendID})
+// }
+// if req.ContentType != 0 {
+// where = append(where, bson.M{"msgs.msg.content_type": req.ContentType})
+// }
+// if req.SessionType != 0 {
+// where = append(where, bson.M{"msgs.msg.session_type": req.SessionType})
+// }
+// if req.SendTime != "" {
+// sendTime, err := time.Parse(time.DateOnly, req.SendTime)
+// if err != nil {
+// return 0, nil, errs.ErrArgs.WrapMsg("invalid sendTime", "req", req.SendTime, "format", time.DateOnly, "cause", err.Error())
+// }
+// where = append(where,
+// bson.M{
+// "msgs.msg.send_time": bson.M{
+// "$gte": sendTime.UnixMilli(),
+// },
+// },
+// bson.M{
+// "msgs.msg.send_time": bson.M{
+// "$lt": sendTime.Add(time.Hour * 24).UnixMilli(),
+// },
+// },
+// )
+// }
+// opt := options.Find().SetLimit(100)
+// res, err := mongoutil.Find[model.MsgDocModel](ctx, m.coll, bson.M{"$and": where}, opt)
+// if err != nil {
+// return 0, nil, err
+// }
+// _ = res
+// fmt.Println()
+//
+// return 0, nil, nil
+// pipeline := bson.A{
+// bson.M{
+// "$unwind": "$msgs",
+// },
+// }
+// if len(where) > 0 {
+// pipeline = append(pipeline, bson.M{
+// "$match": bson.M{"$and": where},
+// })
+// }
+// pipeline = append(pipeline,
+// bson.M{
+// "$project": bson.M{
+// "_id": 0,
+// "msg": "$msgs.msg",
+// },
+// },
+// bson.M{
+// "$count": "count",
+// },
+// )
+// //count, err := mongoutil.Aggregate[int32](ctx, m.coll, pipeline)
+// //if err != nil {
+// // return 0, nil, err
+// //}
+// //if len(count) == 0 || count[0] == 0 {
+// // return 0, nil, nil
+// //}
+// count := []int32{0}
+// pipeline = pipeline[:len(pipeline)-1]
+// pipeline = append(pipeline,
+// bson.M{
+// "$skip": (req.Pagination.GetPageNumber() - 1) * req.Pagination.GetShowNumber(),
+// },
+// bson.M{
+// "$limit": req.Pagination.GetShowNumber(),
+// },
+// )
+// msgs, err := mongoutil.Aggregate[*model.MsgInfoModel](ctx, m.coll, pipeline)
+// if err != nil {
+// return 0, nil, err
+// }
+// for i := range msgs {
+// msgInfo := msgs[i]
+// if msgInfo == nil || msgInfo.Msg == nil {
+// continue
+// }
+// if msgInfo.Revoke != nil {
+// revokeContent := sdkws.MessageRevokedContent{
+// RevokerID: msgInfo.Revoke.UserID,
+// RevokerRole: msgInfo.Revoke.Role,
+// ClientMsgID: msgInfo.Msg.ClientMsgID,
+// RevokerNickname: msgInfo.Revoke.Nickname,
+// RevokeTime: msgInfo.Revoke.Time,
+// SourceMessageSendTime: msgInfo.Msg.SendTime,
+// SourceMessageSendID: msgInfo.Msg.SendID,
+// SourceMessageSenderNickname: msgInfo.Msg.SenderNickname,
+// SessionType: msgInfo.Msg.SessionType,
+// Seq: msgInfo.Msg.Seq,
+// Ex: msgInfo.Msg.Ex,
+// }
+// data, err := jsonutil.JsonMarshal(&revokeContent)
+// if err != nil {
+// return 0, nil, errs.WrapMsg(err, "json.Marshal revokeContent")
+// }
+// elem := sdkws.NotificationElem{Detail: string(data)}
+// content, err := jsonutil.JsonMarshal(&elem)
+// if err != nil {
+// return 0, nil, errs.WrapMsg(err, "json.Marshal elem")
+// }
+// msgInfo.Msg.ContentType = constant.MsgRevokeNotification
+// msgInfo.Msg.Content = string(content)
+// }
+// }
+// //start := (req.Pagination.PageNumber - 1) * req.Pagination.ShowNumber
+// //n := int32(len(msgs))
+// //if start >= n {
+// // return n, []*relation.MsgInfoModel{}, nil
+// //}
+// //if start+req.Pagination.ShowNumber < n {
+// // msgs = msgs[start : start+req.Pagination.ShowNumber]
+// //} else {
+// // msgs = msgs[start:]
+// //}
+// return count[0], msgs, nil
+//}
+
func (m *MsgMgo) RangeUserSendCount(ctx context.Context, start time.Time, end time.Time, group bool, ase bool, pageNumber int32, showNumber int32) (msgCount int64, userCount int64, users []*model.UserCount, dateCount map[string]int64, err error) {
var sort int
if ase {
diff --git a/pkg/common/storage/database/mgo/msg_test.go b/pkg/common/storage/database/mgo/msg_test.go
new file mode 100644
index 000000000..5aed4dc51
--- /dev/null
+++ b/pkg/common/storage/database/mgo/msg_test.go
@@ -0,0 +1,75 @@
+package mgo
+
+import (
+ "context"
+ "github.com/openimsdk/open-im-server/v3/pkg/common/storage/model"
+ "github.com/openimsdk/protocol/msg"
+ "github.com/openimsdk/protocol/sdkws"
+ "github.com/openimsdk/tools/db/mongoutil"
+ "go.mongodb.org/mongo-driver/bson"
+ "go.mongodb.org/mongo-driver/mongo"
+ "go.mongodb.org/mongo-driver/mongo/options"
+ "math/rand"
+ "strconv"
+ "testing"
+ "time"
+)
+
+func TestName1(t *testing.T) {
+ ctx, cancel := context.WithTimeout(context.Background(), time.Second*300)
+ defer cancel()
+ cli := Result(mongo.Connect(ctx, options.Client().ApplyURI("mongodb://openIM:openIM123@172.16.8.48:37017/openim_v3?maxPoolSize=100").SetConnectTimeout(5*time.Second)))
+
+ v := &MsgMgo{
+ coll: cli.Database("openim_v3").Collection("msg3"),
+ }
+
+ req := &msg.SearchMessageReq{
+ //RecvID: "3187706596",
+ //SendID: "7009965934",
+ ContentType: 101,
+ //SendTime: "2024-05-06",
+ //SessionType: 3,
+ Pagination: &sdkws.RequestPagination{
+ PageNumber: 1,
+ ShowNumber: 10,
+ },
+ }
+ total, res, err := v.SearchMessage(ctx, req)
+ if err != nil {
+ panic(err)
+ }
+
+ for i, re := range res {
+ t.Logf("%d => %d | %+v", i+1, re.Msg.Seq, re.Msg.Content)
+ }
+
+ t.Log(total)
+}
+
+func TestName10(t *testing.T) {
+ ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
+ defer cancel()
+ cli := Result(mongo.Connect(ctx, options.Client().ApplyURI("mongodb://openIM:openIM123@172.16.8.48:37017/openim_v3?maxPoolSize=100").SetConnectTimeout(5*time.Second)))
+
+ v := &MsgMgo{
+ coll: cli.Database("openim_v3").Collection("msg3"),
+ }
+ opt := options.Find().SetLimit(1000)
+
+ res, err := mongoutil.Find[model.MsgDocModel](ctx, v.coll, bson.M{}, opt)
+ if err != nil {
+ panic(err)
+ }
+ ctx = context.Background()
+ for i := 0; i < 100000; i++ {
+ for j := range res {
+ res[j].DocID = strconv.FormatUint(rand.Uint64(), 10) + ":0"
+ }
+ if err := mongoutil.InsertMany(ctx, v.coll, res); err != nil {
+ panic(err)
+ }
+ t.Log("====>", time.Now(), i)
+ }
+
+}
diff --git a/pkg/common/storage/database/mgo/seq_conversation_test.go b/pkg/common/storage/database/mgo/seq_conversation_test.go
index 5167314da..42507a693 100644
--- a/pkg/common/storage/database/mgo/seq_conversation_test.go
+++ b/pkg/common/storage/database/mgo/seq_conversation_test.go
@@ -26,7 +26,7 @@ func Mongodb() *mongo.Database {
func TestUserSeq(t *testing.T) {
uSeq := Result(NewSeqUserMongo(Mongodb())).(*seqUserMongo)
- t.Log(uSeq.SetMinSeq(context.Background(), "1000", "2000", 4))
+ t.Log(uSeq.SetUserMinSeq(context.Background(), "1000", "2000", 4))
}
func TestConversationSeq(t *testing.T) {
@@ -35,3 +35,8 @@ func TestConversationSeq(t *testing.T) {
t.Log(cSeq.Malloc(context.Background(), "2000", 10))
t.Log(cSeq.GetMaxSeq(context.Background(), "2000"))
}
+
+func TestUserGetUserReadSeqs(t *testing.T) {
+ uSeq := Result(NewSeqUserMongo(Mongodb())).(*seqUserMongo)
+ t.Log(uSeq.GetUserReadSeqs(context.Background(), "2110910952", []string{"sg_345762580", "2000", "3000"}))
+}
diff --git a/pkg/common/storage/database/mgo/seq_user.go b/pkg/common/storage/database/mgo/seq_user.go
index 7c9a8f133..9faad416a 100644
--- a/pkg/common/storage/database/mgo/seq_user.go
+++ b/pkg/common/storage/database/mgo/seq_user.go
@@ -88,6 +88,14 @@ func (s *seqUserMongo) GetUserReadSeq(ctx context.Context, conversationID string
return s.getSeq(ctx, conversationID, userID, "read_seq")
}
+func (s *seqUserMongo) notFoundSet0(seq map[string]int64, conversationIDs []string) {
+ for _, conversationID := range conversationIDs {
+ if _, ok := seq[conversationID]; !ok {
+ seq[conversationID] = 0
+ }
+ }
+}
+
func (s *seqUserMongo) GetUserReadSeqs(ctx context.Context, userID string, conversationID []string) (map[string]int64, error) {
if len(conversationID) == 0 {
return map[string]int64{}, nil
@@ -102,6 +110,7 @@ func (s *seqUserMongo) GetUserReadSeqs(ctx context.Context, userID string, conve
for _, seq := range seqs {
res[seq.ConversationID] = seq.ReadSeq
}
+ s.notFoundSet0(res, conversationID)
return res, nil
}
diff --git a/pkg/common/storage/database/msg.go b/pkg/common/storage/database/msg.go
index b402f3ac7..84f3a9e3e 100644
--- a/pkg/common/storage/database/msg.go
+++ b/pkg/common/storage/database/msg.go
@@ -37,7 +37,7 @@ type Msg interface {
GetMsgDocModelByIndex(ctx context.Context, conversationID string, index, sort int64) (*model.MsgDocModel, error)
DeleteMsgsInOneDocByIndex(ctx context.Context, docID string, indexes []int) error
MarkSingleChatMsgsAsRead(ctx context.Context, userID string, docID string, indexes []int64) error
- SearchMessage(ctx context.Context, req *msg.SearchMessageReq) (int32, []*model.MsgInfoModel, error)
+ SearchMessage(ctx context.Context, req *msg.SearchMessageReq) (int64, []*model.MsgInfoModel, error)
RangeUserSendCount(ctx context.Context, start time.Time, end time.Time, group bool, ase bool, pageNumber int32, showNumber int32) (msgCount int64, userCount int64, users []*model.UserCount, dateCount map[string]int64, err error)
RangeGroupSendCount(ctx context.Context, start time.Time, end time.Time, ase bool, pageNumber int32, showNumber int32) (msgCount int64, userCount int64, groups []*model.GroupCount, dateCount map[string]int64, err error)
ConvertMsgsDocLen(ctx context.Context, conversationIDs []string)
From a837fecda35b6f0237efdcb9f474aefcc0bfa9d6 Mon Sep 17 00:00:00 2001
From: skiffer-git <44203734@qq.com>
Date: Fri, 26 Jul 2024 16:23:45 +0800
Subject: [PATCH 12/17] merge: fix some typos in comments
---
deployments/templates/env-template.yaml | 2 +-
pkg/common/storage/controller/msg.go | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/deployments/templates/env-template.yaml b/deployments/templates/env-template.yaml
index b1fb8b78b..b019f97e5 100644
--- a/deployments/templates/env-template.yaml
+++ b/deployments/templates/env-template.yaml
@@ -129,7 +129,7 @@ REDIS_PORT=${REDIS_PORT}
# Default: REDIS_PASSWORD=openIM123
REDIS_PASSWORD=${REDIS_PASSWORD}
-# Kakfa username to authenticate with the Kafka service.
+# Kafka username to authenticate with the Kafka service.
# KAFKA_USERNAME=${KAFKA_USERNAME}
# Port on which Kafka distributed streaming platform is running.
diff --git a/pkg/common/storage/controller/msg.go b/pkg/common/storage/controller/msg.go
index 4ea74ef69..49268e049 100644
--- a/pkg/common/storage/controller/msg.go
+++ b/pkg/common/storage/controller/msg.go
@@ -721,7 +721,7 @@ func (db *commonMsgDatabase) deleteMsgRecursion(ctx context.Context, conversatio
}
log.ZDebug(ctx, "doc info", "conversationID", conversationID, "index", index, "docID", msgDocModel.DocID, "len", len(msgDocModel.Msg))
if int64(len(msgDocModel.Msg)) > db.msgTable.GetSingleGocMsgNum() {
- log.ZWarn(ctx, "msgs too large", nil, "lenth", len(msgDocModel.Msg), "docID:", msgDocModel.DocID)
+ log.ZWarn(ctx, "msgs too large", nil, "length", len(msgDocModel.Msg), "docID:", msgDocModel.DocID)
}
if msgDocModel.IsFull() && msgDocModel.Msg[len(msgDocModel.Msg)-1].Msg.SendTime+(remainTime*1000) < timeutil.GetCurrentTimestampByMill() {
log.ZDebug(ctx, "doc is full and all msg is expired", "docID", msgDocModel.DocID)
From 0a7992faab11e96b83a4ee4e3777dafca038ab71 Mon Sep 17 00:00:00 2001
From: skiffer-git <72860476+skiffer-git@users.noreply.github.com>
Date: Fri, 26 Jul 2024 16:30:58 +0800
Subject: [PATCH 13/17] Update env.template
---
config/templates/env.template | 1 +
1 file changed, 1 insertion(+)
diff --git a/config/templates/env.template b/config/templates/env.template
index 5a232b2ae..a6bfcc820 100644
--- a/config/templates/env.template
+++ b/config/templates/env.template
@@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+
# -----------------------------------------------------------------------------
# General Configuration
# This section contains general configuration options for the entire environment.
From 6c0c83eb3b5bdd96c135f6ec88f196b8fc1f2f55 Mon Sep 17 00:00:00 2001
From: skiffer-git <72860476+skiffer-git@users.noreply.github.com>
Date: Fri, 26 Jul 2024 16:32:36 +0800
Subject: [PATCH 14/17] Delete config/templates/env.template
---
config/templates/env.template | 238 ----------------------------------
1 file changed, 238 deletions(-)
delete mode 100644 config/templates/env.template
diff --git a/config/templates/env.template b/config/templates/env.template
deleted file mode 100644
index a6bfcc820..000000000
--- a/config/templates/env.template
+++ /dev/null
@@ -1,238 +0,0 @@
-# Copyright © 2024 OpenIM. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-
-# -----------------------------------------------------------------------------
-# General Configuration
-# This section contains general configuration options for the entire environment.
-# These options can be set via environment variables. If both environment variables
-# and settings in this .env file exist, the environment variables take precedence.
-# -----------------------------------------------------------------------------
-# ==========================
-# General Configuration
-# ==========================
-# These settings apply to the overall environment.
-
-# Data storage directory for persistent data.
-# Example: DATA_DIR=/path/to/data
-DATA_DIR=/data/workspaces/open-im-server
-
-# Docker image registry. Uncomment the preferred one.
-# Options: ghcr.io/openimsdk, openim, registry.cn-hangzhou.aliyuncs.com/openimsdk
-# IMAGE_REGISTRY="ghcr.io/openimsdk"
-# IMAGE_REGISTRY="openim"
-# IMAGE_REGISTRY="registry.cn-hangzhou.aliyuncs.com/openimsdk"
-IMAGE_REGISTRY=ghcr.io/openimsdk
-
-# ======================================
-# ========= Network Configuration ======
-# ======================================
-
-# Subnet for the Docker network.
-# Default: DOCKER_BRIDGE_SUBNET=172.28.0.0/16
-DOCKER_BRIDGE_SUBNET=172.28.0.0/16
-
-# Set and specify the IP addresses of some containers. Generally speaking,
-# you do not need to modify these configurations to facilitate debugging
-DOCKER_BRIDGE_GATEWAY=172.28.0.1
-MONGO_NETWORK_ADDRESS=172.28.0.2
-REDIS_NETWORK_ADDRESS=172.28.0.3
-KAFKA_NETWORK_ADDRESS=172.28.0.4
-ZOOKEEPER_NETWORK_ADDRESS=172.28.0.5
-MINIO_NETWORK_ADDRESS=172.28.0.6
-OPENIM_WEB_NETWORK_ADDRESS=172.28.0.7
-OPENIM_SERVER_NETWORK_ADDRESS=172.28.0.8
-OPENIM_CHAT_NETWORK_ADDRESS=172.28.0.9
-PROMETHEUS_NETWORK_ADDRESS=172.28.0.10
-GRAFANA_NETWORK_ADDRESS=172.28.0.11
-NODE_EXPORTER_NETWORK_ADDRESS=172.28.0.12
-OPENIM_ADMIN_FRONT_NETWORK_ADDRESS=172.28.0.13
-ALERT_MANAGER_NETWORK_ADDRESS=172.28.0.14
-
-# ==============================================================================
-# Configuration Update Instructions
-# ==============================================================================
-# This header outlines the methods to update common variables in config.yaml and .env files.
-# These instructions are vital for maintaining the OpenIM environment's configuration.
-#
-# METHOD 1: Regenerate All Configurations
-# ----------------------------------------
-# Use this method to regenerate all configurations.
-# Steps:
-# 1. Delete existing config files:
-# - openim-server/config/config.yaml
-# - openim-chat/config/config.yaml
-# 2. Modify the .env file as required.
-# 3. Run 'docker compose up -d'. This will regenerate:
-# - config/config.yaml
-#
-# METHOD 2: Modify Individual Configuration Files
-# -----------------------------------------------
-# Use this method to update specific configuration files.
-# Steps:
-# 1. Modify the .env file as necessary.
-# 2. Update the corresponding entries in:
-# - config/config.yaml
-# 3. Restart the services with 'docker compose up -d'.
-# 4. Special Note: If you modify OPENIM_IP, API_OPENIM_PORT, or MINIO_PORT in .env,
-# ensure to update the corresponding services and configurations accordingly.
-#
-# It is essential to follow these methods to ensure consistent and correct application behavior.
-# ==============================================================================
-# Local IP address of the service. Modify if necessary.
-# Example: OPENIM_IP=172.28.0.1，
-OPENIM_IP=127.0.0.1
-
-# ----- ZooKeeper Configuration -----
-# Port for ZooKeeper service.
-# Default: ZOOKEEPER_PORT=12181
-ZOOKEEPER_PORT=12181
-
-# MongoDB service port configuration.
-# Default: MONGO_PORT=37017
-MONGO_PORT=37017
-
-# Password for MongoDB admin user. Used for service authentication.
-# Default: MONGO_PASSWORD=openIM123
-MONGO_PASSWORD=openIM123
-
-# Username for a regular OpenIM user in MongoDB.
-# Default: MONGO_OPENIM_USERNAME=openIM
-MONGO_OPENIM_USERNAME=openIM
-
-# Password for a regular OpenIM user in MongoDB.
-# Default: MONGO_OPENIM_PASSWORD=openIM123456
-MONGO_OPENIM_PASSWORD=openIM123
-
-# Specifies the database name to be used within MongoDB.
-# Default: MONGO_DATABASE=openim_v3
-MONGO_DATABASE=openim_v3
-
-MONGO_MAX_POOL_SIZE=100
-# ----- Redis Configuration -----
-
-# Port on which Redis in-memory data structure store is running.
-# Default: REDIS_PORT=16379
-REDIS_PORT=16379
-
-# Password to authenticate with the Redis service.
-# Default: REDIS_PASSWORD=openIM123
-REDIS_PASSWORD=openIM123
-
-# Kafka username to authenticate with the Kafka service.
-# KAFKA_USERNAME=''
-
-# Port on which Kafka distributed streaming platform is running.
-# Default: KAFKA_PORT=19092
-KAFKA_PORT=19094
-
-# Topic in Kafka for storing the latest messages in Redis.
-# Default: KAFKA_LATESTMSG_REDIS_TOPIC=latestMsgToRedis
-KAFKA_LATESTMSG_REDIS_TOPIC=latestMsgToRedis
-
-# MINIO_PORT
-# ----------
-# MINIO_PORT sets the port for the MinIO object storage service.
-# Upon changing this port, the MinIO endpoint URLs in the config/config.yaml file must be updated
-# to reflect this change. The endpoints include both the 'endpoint' and 'signEndpoint'
-# under the MinIO configuration.
-#
-# Default: MINIO_PORT=10005
-MINIO_PORT=10005
-
-# Access key to authenticate with the MinIO service.
-# Default: MINIO_ACCESS_KEY=root
-# MINIO_ACCESS_KEY=root
-
-# Secret key corresponding to the access key for MinIO authentication.
-# Default: MINIO_SECRET_KEY=openIM123
-MINIO_SECRET_KEY=openIM123
-
-# ----- Prometheus Configuration -----
-# Port on which Prometheus service is running.
-# Default: PROMETHEUS_PORT=19090
-PROMETHEUS_PORT=19090
-
-# ----- Grafana Configuration -----
-# Port on which Grafana service is running.
-# Default: GRAFANA_PORT=13000
-GRAFANA_PORT=13000
-
-# ======================================
-# ============ OpenIM Web ===============
-# ======================================
-
-# Port on which OpenIM web service is running.
-# Default: OPENIM_WEB_PORT=11001
-OPENIM_WEB_PORT=11001
-
-# ======================================
-# ========= OpenIM Server ==============
-# ======================================
-# Port for the OpenIM WebSockets.
-# Default: OPENIM_WS_PORT=10001
-OPENIM_WS_PORT=10001
-
-# API_OPENIM_PORT
-# ---------------
-# This variable defines the port on which the OpenIM API service will listen.
-# When changing this port, it's essential to update the apiURL in the config.yaml file
-# to ensure the API service is accessible at the new port.
-#
-# Default: API_OPENIM_PORT=10002
-API_OPENIM_PORT=10002
-
-# ======================================
-# ========== OpenIM Chat ===============
-# ======================================
-
-# Branch name for OpenIM chat.
-# Default: CHAT_IMAGE_VERSION=main
-CHAT_IMAGE_VERSION=main
-
-# Port for the OpenIM chat API.
-# Default: OPENIM_CHAT_API_PORT=10008
-OPENIM_CHAT_API_PORT=10008
-
-# Port for the OpenIM admin API.
-# Default: OPENIM_ADMIN_API_PORT=10009
-OPENIM_ADMIN_API_PORT=10009
-
-# ======================================
-# ========== OpenIM Admin ==============
-# ======================================
-
-# Branch name for OpenIM server.
-# Default: SERVER_IMAGE_VERSION=main
-SERVER_IMAGE_VERSION=main
-
-# Port for the node exporter.
-# Default: NODE_EXPORTER_PORT=19100
-NODE_EXPORTER_PORT=19100
-
-# Port for the prometheus.
-# Default: PROMETHEUS_PORT=19090
-PROMETHEUS_PORT=19090
-
-# Port for the grafana.
-# Default: GRAFANA_PORT=13000
-GRAFANA_PORT=13000
-
-# Port for the admin front.
-# Default: OPENIM_ADMIN_FRONT_PORT=11002
-OPENIM_ADMIN_FRONT_PORT=11002
-
-# Port for the alertmanager.
-# Default: ALERT_MANAGER_PORT=19093
-ALERT_MANAGER_PORT=19093
From e9b3a1952f7ab92846dab345ee97132c4e5b1dcd Mon Sep 17 00:00:00 2001
From: blooming <37789413+Bloomingg@users.noreply.github.com>
Date: Fri, 26 Jul 2024 16:36:11 +0800
Subject: [PATCH 15/17] cicd: add e2e in ci & before build docker image (#2346)
* cicd: add e2e in ci & before build docker image
* cicd: fix env minio addr
* cicd: fix build image timing
---
.github/workflows/build-docker-image.yml | 208 +++++++++++++----------
.github/workflows/link-pr.yml | 53 ------
.github/workflows/openimci.yml | 150 +++++-----------
.github/workflows/stale.yml | 48 ------
4 files changed, 169 insertions(+), 290 deletions(-)
delete mode 100644 .github/workflows/link-pr.yml
delete mode 100644 .github/workflows/stale.yml
diff --git a/.github/workflows/build-docker-image.yml b/.github/workflows/build-docker-image.yml
index b4733116e..63982a012 100644
--- a/.github/workflows/build-docker-image.yml
+++ b/.github/workflows/build-docker-image.yml
@@ -19,8 +19,27 @@ on:
branches:
- main
- release-*
+ paths-ignore:
+ - "docs/**"
+ - "README.md"
+ - "README_zh-CN.md"
+ - "**.md"
+ - "docs/**"
+ - "CONTRIBUTING.md"
tags:
- v*
+ pull_request:
+ types: [closed]
+ branches:
+ - main
+ - release-*
+ paths-ignore:
+ - "docs/**"
+ - "README.md"
+ - "README_zh-CN.md"
+ - "**.md"
+ - "docs/**"
+ - "CONTRIBUTING.md"
workflow_dispatch:
env:
@@ -29,109 +48,112 @@ env:
jobs:
build-dockerhub:
- if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.action == 'closed' && github.event.pull_request.merged == true)
runs-on: ubuntu-latest
+ if: ${{ !(github.event_name == 'pull_request' && github.event.pull_request.merged == false) }}
steps:
- - name: Checkout
+ - name: Checkout main repository
uses: actions/checkout@v4
+ with:
+ path: main-repo
+
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-# docker.io/openim/openim-server:latest
- - name: Extract metadata (tags, labels) for Docker
- id: meta
- uses: docker/metadata-action@v5.5.1
+ - name: Build and push Docker image
+ uses: docker/build-push-action@v5
with:
- images: openim/openim-server
- # generate Docker tags based on the following events/attributes
- tags: |
- type=ref,event=tag
- type=schedule
- type=ref,event=branch
- type=ref,event=pr
- type=semver,pattern={{version}}
- type=semver,pattern=v{{version}}
- type=semver,pattern={{major}}.{{minor}}
- type=semver,pattern={{major}}
- type=sha
+ context: ./main-repo
+ load: true
+ tags: "openim/openim-server:local"
- - name: Log in to Docker Hub
- uses: docker/login-action@v3
+ - name: Checkout compose repository
+ uses: actions/checkout@v4
with:
- username: ${{ secrets.DOCKER_USERNAME }}
- password: ${{ secrets.DOCKER_PASSWORD }}
+ repository: "openimsdk/openim-docker"
+ path: "compose-repo"
- - name: Build and push Docker image
- uses: docker/build-push-action@v5
- with:
- context: .
- # linux/ppc64le,linux/s390x
- platforms: linux/amd64,linux/arm64
- push: ${{ github.event_name != 'pull_request' }}
- tags: ${{ steps.meta.outputs.tags }}
- labels: ${{ steps.meta.outputs.labels }}
+ - name: Get Internal IP Address
+ id: get-ip
+ run: |
+ IP=$(hostname -I | awk '{print $1}')
+ echo "The IP Address is: $IP"
+ echo "::set-output name=ip::$IP"
- build-aliyun:
- runs-on: ubuntu-latest
- steps:
- - name: Checkout
+ - name: Update .env to use the local image
+ run: |
+ sed -i 's|OPENIM_SERVER_IMAGE=.*|OPENIM_SERVER_IMAGE=openim/openim-server:local|' ${{ github.workspace }}/compose-repo/.env
+ sed -i 's|MINIO_EXTERNAL_ADDRESS=.*|MINIO_EXTERNAL_ADDRESS=http://${{ steps.get-ip.outputs.ip }}:10005|' ${{ github.workspace }}/compose-repo/.env
+
+ - name: Start services using Docker Compose
+ run: |
+ cd ${{ github.workspace }}/compose-repo
+ docker compose up -d
+ sleep 30
+
+ - name: Check openim-server health
+ run: |
+ timeout=300
+ interval=30
+ elapsed=0
+ while [[ $elapsed -le $timeout ]]; do
+ if ! docker exec openim-server mage check; then
+ echo "openim-server is not ready, waiting..."
+ sleep $interval
+ elapsed=$(($elapsed + $interval))
+ else
+ echo "Health check successful"
+ exit 0
+ fi
+ done
+ echo "Health check failed after 5 minutes"
+ exit 1
+
+ - name: Check openim-chat health
+ if: success()
+ run: |
+ if ! docker exec openim-chat mage check; then
+ echo "openim-chat check failed"
+ exit 1
+ else
+ echo "Health check successful"
+ exit 0
+ fi
+
+ - name: Checkout e2e
+ if: success()
uses: actions/checkout@v4
- - name: Set up QEMU
- uses: docker/setup-qemu-action@v3
- - name: Set up Docker Buildx
- uses: docker/setup-buildx-action@v3
-# registry.cn-hangzhou.aliyuncs.com/openimsdk/openim-server:latest
- - name: Extract metadata (tags, labels) for Docker
- id: meta2
- uses: docker/metadata-action@v5.5.1
with:
- images: registry.cn-hangzhou.aliyuncs.com/openimsdk/openim-server
- # generate Docker tags based on the following events/attributes
- tags: |
- type=ref,event=tag
- type=schedule
- type=ref,event=branch
- type=ref,event=pr
- type=semver,pattern={{version}}
- type=semver,pattern=v{{version}}
- type=semver,pattern={{major}}.{{minor}}
- type=semver,pattern={{major}}
- type=sha
+ repository: "openimsdk/test-e2e"
+ path: e2e-repo
- - name: Log in to AliYun Docker Hub
- uses: docker/login-action@v3
+ - name: Set up Python 3.9
+ uses: actions/setup-python@v4
with:
- registry: registry.cn-hangzhou.aliyuncs.com
- username: ${{ secrets.ALIREGISTRY_USERNAME }}
- password: ${{ secrets.ALIREGISTRY_TOKEN }}
+ python-version: '3.9'
- - name: Build and push Docker image
- uses: docker/build-push-action@v5
- with:
- context: .
- # linux/ppc64le,linux/s390x
- platforms: linux/amd64,linux/arm64
- push: ${{ github.event_name != 'pull_request' }}
- tags: ${{ steps.meta2.outputs.tags }}
- labels: ${{ steps.meta2.outputs.labels }}
+ - name: Install dependencies
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y xvfb libxi6 libgconf-2-4
+ cd ${{ github.workspace }}/e2e-repo
+ pip install -r requirements.txt
+
+ - name: Run tests
+ run: |
+ cd ${{ github.workspace }}/e2e-repo
+ xvfb-run --auto-servernum --server-args='-screen 0 1920x1080x24' pytest -v -s ./script
- build-ghcr:
- runs-on: ubuntu-latest
- steps:
- - name: Checkout
- uses: actions/checkout@v4
- - name: Set up QEMU
- uses: docker/setup-qemu-action@v3
- - name: Set up Docker Buildx
- uses: docker/setup-buildx-action@v3
-# ghcr.io/openimsdk/openim-server:latest
- name: Extract metadata (tags, labels) for Docker
- id: meta3
+ if: success()
+ id: meta
uses: docker/metadata-action@v5.5.1
with:
- images: ghcr.io/openimsdk/openim-server
+ images: |
+ openim/openim-server
+ ghcr.io/openimsdk/openim-server
+ registry.cn-hangzhou.aliyuncs.com/openimsdk/openim-server
# generate Docker tags based on the following events/attributes
tags: |
type=ref,event=tag
@@ -144,19 +166,33 @@ jobs:
type=semver,pattern={{major}}
type=sha
+ - name: Log in to Docker Hub
+ uses: docker/login-action@v2
+ with:
+ username: ${{ secrets.DOCKER_USERNAME }}
+ password: ${{ secrets.DOCKER_PASSWORD }}
+
- name: Log in to GitHub Container Registry
- uses: docker/login-action@v3
+ uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
+ - name: Log in to Aliyun Container Registry
+ uses: docker/login-action@v2
+ with:
+ registry: registry.cn-hangzhou.aliyuncs.com
+ username: ${{ secrets.ALIREGISTRY_USERNAME }}
+ password: ${{ secrets.ALIREGISTRY_TOKEN }}
+
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
- context: .
+ context: ./main-repo
+ push: true
# linux/ppc64le,linux/s390x
platforms: linux/amd64,linux/arm64
- push: ${{ github.event_name != 'pull_request' }}
- tags: ${{ steps.meta3.outputs.tags }}
- labels: ${{ steps.meta3.outputs.labels }}
+ tags: ${{ steps.meta.outputs.tags }}
+ labels: ${{ steps.meta.outputs.labels }}
+
diff --git a/.github/workflows/link-pr.yml b/.github/workflows/link-pr.yml
deleted file mode 100644
index 6c5ff6c5b..000000000
--- a/.github/workflows/link-pr.yml
+++ /dev/null
@@ -1,53 +0,0 @@
-# Copyright © 2023 OpenIM open source community. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: Github Rebot for Link check error
-
-# Every Monday at 12:30 p.m
-on:
- schedule:
- - cron: '30 12 * * 1'
-
-jobs:
- linkChecker:
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v4
-
- - name: Link Checker
- id: lychee
- uses: lycheeverse/lychee-action@v1.9.3
- with:
- # For parameter description, see https://github.com/lycheeverse/lychee#commandline-parameters
- # Actions Link address -> https://github.com/lycheeverse/lychee-action
- # -E, --exclude-all-private Exclude all private IPs from checking.
- # -i, --insecure Proceed for server connections considered insecure (invalid TLS)
- # -n, --no-progress Do not show progress bar.
- # -t, --timeout Website timeout in seconds from connect to response finished [default:20]
- # --max-concurrency Maximum number of concurrent network requests [default: 128]
- # -a --accept Comma-separated list of accepted status codes for valid links
- # docs/.vitepress/dist the site directory to check
- # ./*.md all markdown files in the root directory
- args: --verbose -E -i --no-progress --exclude-path './CHANGELOG' './**/*.md'
- env:
- GITHUB_TOKEN: ${{secrets.BOT_GITHUB_TOKEN}}
-
- - name: Create Issue From File
- if: env.lychee_exit_code != 0
- uses: peter-evans/create-issue-from-file@v5
- with:
- title: Bug reports for links in OpenIM docs
- content-filepath: ./lychee/out.md
- labels: kind/documentation, triage/unresolved, report
- token: ${{ secrets.BOT_GITHUB_TOKEN }}
diff --git a/.github/workflows/openimci.yml b/.github/workflows/openimci.yml
index 83d495a0e..9952a1381 100644
--- a/.github/workflows/openimci.yml
+++ b/.github/workflows/openimci.yml
@@ -71,6 +71,18 @@ jobs:
run: sudo bash bootstrap.sh
timeout-minutes: 20
+ - name: Get Internal IP Address
+ id: get-ip
+ run: |
+ IP=$(hostname -I | awk '{print $1}')
+ echo "The IP Address is: $IP"
+ echo "::set-output name=ip::$IP"
+
+ - name: Update .env
+ run: |
+ sed -i 's|externalAddress:.*|externalAddress: "http://${{ steps.get-ip.outputs.ip }}:10005"|' config/minio.yml
+ cat config/minio.yml
+
- name: Build, Start, Check Services and Print Logs for Linux
run: |
sudo mage
@@ -84,108 +96,40 @@ jobs:
sudo mage start
sudo mage check
+ - name: Checkout chat repository
+ uses: actions/checkout@v4
+ with:
+ repository: 'openimsdk/chat'
+ path: 'chat-repo'
-# build-mac:
-# name: Execute OpenIM Script On macOS
-# runs-on: macos-latest
-# permissions:
-# contents: write
-# pull-requests: write
-# environment:
-# name: openim
-# strategy:
-# matrix:
-# arch: [arm64, armv7, amd64]
-#
-# steps:
-# - uses: actions/checkout@v3
-
-# - name: Set up Go
-# uses: actions/setup-go@v4
-# with:
-# go-version: '1.21'
-
-
-# while ! docker system info > /dev/null 2>&1; do
-# echo "Waiting for Docker to start..."
-# sleep 10 # Increased delay to ensure Docker starts properly
-# done
-
-# - name: Install Docker
-# run: |
-# brew install docker
-# brew install docker-compose
-# sleep 10
-# docker-compose up -d
-# sleep 30
-# timeout-minutes: 20
-#
-
-# - name: init
-# run: sudo bash bootstrap.sh
-# timeout-minutes: 20
-
-# - name: Build, Start, Check Services and Print Logs for Linux
-# run: |
-# sudo mage
-# sudo mage start
-# sudo mage check
-
-# - name: Restart Services and Print Logs
-# run: |
-# sudo mage stop
-# sudo mage start
-# sudo mage check
-
-# build-windows:
-# name: Execute OpenIM Script On Windows
-# runs-on: windows-latest
-# permissions:
-# contents: write
-# pull-requests: write
-# environment:
-# name: openim
-# strategy:
-# matrix:
-# arch: [arm64, armv7, amd64]
-#
-# steps:
-# - uses: actions/checkout@v3
-
-# - name: Set up Go
-# uses: actions/setup-go@v4
-# with:
-# go-version: '1.21'
-
-# - name: Set up Docker for Windows
-# run: |
-# $images = @("zookeeper", "redis", "kafka")
-# foreach ($image in $images) {
-# $tag = "$image:latest"
-# docker pull $tag | Out-Null
-# if ($LASTEXITCODE -ne 0) {
-# Write-Host "Skipping $image as it is not available for Windows"
-# } else {
-# Write-Host "Successfully pulled $image"
-# }
-# }
-# docker compose up -d
-# Start-Sleep -Seconds 30
-# timeout-minutes: 20
-# shell: pwsh
-
-# - name: init
-# run: bootstrap.bat
-# timeout-minutes: 20
-
-# - name: Build, Start, Check Services and Print Logs for Linux
-# run: |
-# mage
-# mage start
-# mage check
+ - name: Build and Start Chat Services
+ run: |
+ cd ${{ github.workspace }}/chat-repo
+ sudo mage
+ sudo mage start
+ sudo mage check
+
+ - name: Checkout e2e repository
+ uses: actions/checkout@v4
+ with:
+ repository: "openimsdk/test-e2e"
+ path: e2e-repo
-# - name: Restart Services and Print Logs
-# run: |
-# mage stop
-# mage start
-# mage check
+ - name: Set up Python 3.9
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.9'
+
+ - name: Install dependencies
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y xvfb libxi6 libgconf-2-4
+ cd ${{ github.workspace }}/e2e-repo
+ pip install -r requirements.txt
+
+ - name: Run tests
+ run: |
+ cd ${{ github.workspace }}/e2e-repo
+ xvfb-run --auto-servernum --server-args='-screen 0 1920x1080x24' pytest -v -s ./script
+
+
\ No newline at end of file
diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml
deleted file mode 100644
index 4445f6dda..000000000
--- a/.github/workflows/stale.yml
+++ /dev/null
@@ -1,48 +0,0 @@
-# Copyright © 2023 OpenIM. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# This workflow warns and then closes issues and PRs that have had no activity for a specified amount of time.
-#
-# You can adjust the behavior by modifying this file.
-# For more information, see:
-# https://github.com/actions/stale
-name: Mark stale issues and pull requests
-
-on:
- schedule:
- - cron: '0 8 * * 1'
-
-jobs:
- stale:
-
- runs-on: ubuntu-latest
- permissions:
- issues: write
- pull-requests: write
-
- steps:
- - uses: actions/stale@v9
- with:
- repo-token: ${{ secrets.BOT_GITHUB_TOKEN }}
- days-before-stale: 60
- days-before-close: 305
- stale-issue-message: 'This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.'
- stale-pr-message: 'This issue is stale because it has been open 60 days with no activity.'
- close-issue-message: 'This issue was closed because it has been stalled for 7 days with no activity.'
- close-pr-message: 'This PR was closed because it has been stalled for 7 days with no activity. You can reopen it if you want.'
- stale-pr-label: lifecycle/stale
- stale-issue-label: lifecycle/stale
- exempt-issue-labels: 'openim'
- exempt-pr-labels: 'openim'
- exempt-draft-pr: true
From ff66e972219a50e9065da24a5756269fcb0b861c Mon Sep 17 00:00:00 2001
From: blooming <37789413+Bloomingg@users.noreply.github.com>
Date: Fri, 26 Jul 2024 18:20:54 +0800
Subject: [PATCH 16/17] fix: rm e2e in ci (#2449)
---
.github/workflows/build-docker-image.yml | 42 ++++++++---------
.github/workflows/openimci.yml | 58 ++++++++++++------------
2 files changed, 50 insertions(+), 50 deletions(-)
diff --git a/.github/workflows/build-docker-image.yml b/.github/workflows/build-docker-image.yml
index 63982a012..d0b9dddbc 100644
--- a/.github/workflows/build-docker-image.yml
+++ b/.github/workflows/build-docker-image.yml
@@ -121,29 +121,29 @@ jobs:
exit 0
fi
- - name: Checkout e2e
- if: success()
- uses: actions/checkout@v4
- with:
- repository: "openimsdk/test-e2e"
- path: e2e-repo
-
- - name: Set up Python 3.9
- uses: actions/setup-python@v4
- with:
- python-version: '3.9'
+ # - name: Checkout e2e
+ # if: success()
+ # uses: actions/checkout@v4
+ # with:
+ # repository: "openimsdk/test-e2e"
+ # path: e2e-repo
+
+ # - name: Set up Python 3.9
+ # uses: actions/setup-python@v4
+ # with:
+ # python-version: '3.9'
- - name: Install dependencies
- run: |
- sudo apt-get update
- sudo apt-get install -y xvfb libxi6 libgconf-2-4
- cd ${{ github.workspace }}/e2e-repo
- pip install -r requirements.txt
+ # - name: Install dependencies
+ # run: |
+ # sudo apt-get update
+ # sudo apt-get install -y xvfb libxi6 libgconf-2-4
+ # cd ${{ github.workspace }}/e2e-repo
+ # pip install -r requirements.txt
- - name: Run tests
- run: |
- cd ${{ github.workspace }}/e2e-repo
- xvfb-run --auto-servernum --server-args='-screen 0 1920x1080x24' pytest -v -s ./script
+ # - name: Run tests
+ # run: |
+ # cd ${{ github.workspace }}/e2e-repo
+ # xvfb-run --auto-servernum --server-args='-screen 0 1920x1080x24' pytest -v -s ./script
- name: Extract metadata (tags, labels) for Docker
if: success()
diff --git a/.github/workflows/openimci.yml b/.github/workflows/openimci.yml
index 9952a1381..8f3630dd0 100644
--- a/.github/workflows/openimci.yml
+++ b/.github/workflows/openimci.yml
@@ -71,17 +71,17 @@ jobs:
run: sudo bash bootstrap.sh
timeout-minutes: 20
- - name: Get Internal IP Address
- id: get-ip
- run: |
- IP=$(hostname -I | awk '{print $1}')
- echo "The IP Address is: $IP"
- echo "::set-output name=ip::$IP"
+ # - name: Get Internal IP Address
+ # id: get-ip
+ # run: |
+ # IP=$(hostname -I | awk '{print $1}')
+ # echo "The IP Address is: $IP"
+ # echo "::set-output name=ip::$IP"
- - name: Update .env
- run: |
- sed -i 's|externalAddress:.*|externalAddress: "http://${{ steps.get-ip.outputs.ip }}:10005"|' config/minio.yml
- cat config/minio.yml
+ # - name: Update .env
+ # run: |
+ # sed -i 's|externalAddress:.*|externalAddress: "http://${{ steps.get-ip.outputs.ip }}:10005"|' config/minio.yml
+ # cat config/minio.yml
- name: Build, Start, Check Services and Print Logs for Linux
run: |
@@ -109,27 +109,27 @@ jobs:
sudo mage start
sudo mage check
- - name: Checkout e2e repository
- uses: actions/checkout@v4
- with:
- repository: "openimsdk/test-e2e"
- path: e2e-repo
+ # - name: Checkout e2e repository
+ # uses: actions/checkout@v4
+ # with:
+ # repository: "openimsdk/test-e2e"
+ # path: e2e-repo
- - name: Set up Python 3.9
- uses: actions/setup-python@v4
- with:
- python-version: '3.9'
+ # - name: Set up Python 3.9
+ # uses: actions/setup-python@v4
+ # with:
+ # python-version: '3.9'
- - name: Install dependencies
- run: |
- sudo apt-get update
- sudo apt-get install -y xvfb libxi6 libgconf-2-4
- cd ${{ github.workspace }}/e2e-repo
- pip install -r requirements.txt
+ # - name: Install dependencies
+ # run: |
+ # sudo apt-get update
+ # sudo apt-get install -y xvfb libxi6 libgconf-2-4
+ # cd ${{ github.workspace }}/e2e-repo
+ # pip install -r requirements.txt
- - name: Run tests
- run: |
- cd ${{ github.workspace }}/e2e-repo
- xvfb-run --auto-servernum --server-args='-screen 0 1920x1080x24' pytest -v -s ./script
+ # - name: Run tests
+ # run: |
+ # cd ${{ github.workspace }}/e2e-repo
+ # xvfb-run --auto-servernum --server-args='-screen 0 1920x1080x24' pytest -v -s ./script
\ No newline at end of file
From 220a01d7f8d99fc2c1fb2cddca69ba44e96c2150 Mon Sep 17 00:00:00 2001
From: printlin <32053356+printlin@users.noreply.github.com>
Date: Fri, 26 Jul 2024 18:24:25 +0800
Subject: [PATCH 17/17] fix: fill notification offlinePush by config (#2422)
* fix: fill notification offlinePush by config
* fix: fill notification OfflinePush by config
---
pkg/rpcclient/msg.go | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/pkg/rpcclient/msg.go b/pkg/rpcclient/msg.go
index f660c74dd..124cc49af 100644
--- a/pkg/rpcclient/msg.go
+++ b/pkg/rpcclient/msg.go
@@ -324,6 +324,10 @@ func (s *NotificationSender) send(ctx context.Context, sendID, recvID string, co
options := config.GetOptionsByNotification(optionsConfig)
s.SetOptionsByContentType(ctx, options, contentType)
msg.Options = options
+ // fill Notification OfflinePush by config
+ offlineInfo.Title = optionsConfig.OfflinePush.Title
+ offlineInfo.Desc = optionsConfig.OfflinePush.Desc
+ offlineInfo.Ex = optionsConfig.OfflinePush.Ext
msg.OfflinePushInfo = &offlineInfo
req.MsgData = &msg
_, err = s.sendMsg(ctx, &req)