You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Open-IM-Server/pkg/rpccache/conversation.go

171 lines
7.1 KiB

// Copyright © 2024 OpenIM. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
package rpccache
import (
"context"
"github.com/openimsdk/open-im-server/v3/pkg/common/config"
"github.com/openimsdk/open-im-server/v3/pkg/common/storage/cache/cachekey"
3.6.1 code conventions (#2202) * refactor: webhooks update. * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * feat: s3 api addr * refactor: webhooks update. * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * refactor: webhooks update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * refactor: kafka update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Windows can compile and run. * Windows can compile and run. * refactor: kafka update. * feat: msg cache split * refactor: webhooks update * refactor: webhooks update * refactor: friends update * refactor: group update * refactor: third update * refactor: api update * refactor: crontab update * refactor: msggateway update * mage * mage * refactor: all module update. * check * refactor: all module update. * load config * load config * load config * load config * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update tools * update tools * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update protocol * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: api remove token auth by redis directly. * Code Refactoring * refactor: websocket auth change to call rpc of auth. * refactor: kick online user and remove token change to call auth rpc. * refactor: kick online user and remove token change to call auth rpc. * refactor: remove msggateway redis. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * fix: runtime: goroutine stack exceeds * refactor: cmd update. * refactor notification * refactor notification * refactor * refactor: cmd update. * refactor: cmd update. * refactor * refactor * refactor * protojson * protojson * protojson * go mod * wrapperspb * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: context update. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: api name change. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: update file * refactor * refactor * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: callback update. * fix: callback update. * refactor * fix: update message. * fix: msg cache timeout. * refactor * refactor * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * refactor * refactor * fix: push update. * fix: websocket handle error remove when upgrade error. * fix: priority url * fix: minio config * refactor: add zk logger. * refactor * fix: minio config * refactor * refactor * refactor * refactor * refactor: remove zk logger. * refactor: update tools version. * refactor * refactor: update server version to 3.7.0. * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor: zk log debug. * refactor * refactor * refactor * refactor: log level change. * refactor: 3.7.0 code conventions. --------- Co-authored-by: skiffer-git <44203734@qq.com> Co-authored-by: withchao <993506633@qq.com>
7 months ago
"github.com/openimsdk/open-im-server/v3/pkg/localcache"
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
"github.com/openimsdk/open-im-server/v3/pkg/rpcclient"
refactor: 3.7.0 code conventions. (#2148) * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * feat: add code lint * feat: add code lint * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * feat: code format * Script Refactoring * Script Refactoring * Script Refactoring * Adjust MinIO configuration settings * Adjust configuration settings * Adjust configuration settings * refactor: config change. * refactor: webhooks update. * Adjust configuration settings * refactor: webhooks update. * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * feat: s3 api addr * refactor: webhooks update. * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * refactor: webhooks update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * refactor: kafka update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Windows can compile and run. * Windows can compile and run. * refactor: kafka update. * feat: msg cache split * refactor: webhooks update * refactor: webhooks update * refactor: friends update * refactor: group update * refactor: third update * refactor: api update * refactor: crontab update * refactor: msggateway update * mage * mage * refactor: all module update. * check * refactor: all module update. * load config * load config * load config * load config * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update tools * update tools * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update protocol * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: api remove token auth by redis directly. * Code Refactoring * refactor: websocket auth change to call rpc of auth. * refactor: kick online user and remove token change to call auth rpc. * refactor: kick online user and remove token change to call auth rpc. * refactor: remove msggateway redis. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * fix: runtime: goroutine stack exceeds * refactor: cmd update. * refactor notification * refactor notification * refactor * refactor: cmd update. * refactor: cmd update. * refactor * refactor * refactor * protojson * protojson * protojson * go mod * wrapperspb * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: context update. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: api name change. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: update file * refactor * refactor * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: callback update. * fix: callback update. * refactor * fix: update message. * fix: msg cache timeout. * refactor * refactor * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: websocket handle error remove when upgrade error. --------- Co-authored-by: skiffer-git <44203734@qq.com> Co-authored-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> Co-authored-by: withchao <993506633@qq.com>
7 months ago
pbconversation "github.com/openimsdk/protocol/conversation"
"github.com/openimsdk/tools/errs"
"github.com/openimsdk/tools/log"
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
"github.com/openimsdk/tools/utils/datautil"
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
"github.com/redis/go-redis/v9"
"golang.org/x/sync/errgroup"
)
const (
conversationWorkerCount = 20
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
)
refactor: 3.7.0 code conventions. (#2148) * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * feat: add code lint * feat: add code lint * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * Script Refactoring * feat: code format * Script Refactoring * Script Refactoring * Script Refactoring * Adjust MinIO configuration settings * Adjust configuration settings * Adjust configuration settings * refactor: config change. * refactor: webhooks update. * Adjust configuration settings * refactor: webhooks update. * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * feat: s3 api addr * refactor: webhooks update. * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * Adjust configuration settings * refactor: webhooks update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * refactor: kafka update. * refactor: kafka update. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Simplify the Docker Compose configuration, remove unnecessary environment variables, and eliminate the gateway service. * Windows can compile and run. * Windows can compile and run. * refactor: kafka update. * feat: msg cache split * refactor: webhooks update * refactor: webhooks update * refactor: friends update * refactor: group update * refactor: third update * refactor: api update * refactor: crontab update * refactor: msggateway update * mage * mage * refactor: all module update. * check * refactor: all module update. * load config * load config * load config * load config * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * refactor: all module update. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update tools * update tools * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * update protocol * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: all module update. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * Optimize Docker configuration and script. * refactor: api remove token auth by redis directly. * Code Refactoring * refactor: websocket auth change to call rpc of auth. * refactor: kick online user and remove token change to call auth rpc. * refactor: kick online user and remove token change to call auth rpc. * refactor: remove msggateway redis. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor webhook * refactor: cmd update. * refactor: cmd update. * fix: runtime: goroutine stack exceeds * refactor: cmd update. * refactor notification * refactor notification * refactor * refactor: cmd update. * refactor: cmd update. * refactor * refactor * refactor * protojson * protojson * protojson * go mod * wrapperspb * refactor: cmd update. * refactor: cmd update. * refactor: cmd update. * refactor: context update. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: websocket update info. * refactor: api name change. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: update file * refactor * refactor * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * refactor: debug info. * fix: callback update. * fix: callback update. * refactor * fix: update message. * fix: msg cache timeout. * refactor * refactor * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: push update. * fix: websocket handle error remove when upgrade error. --------- Co-authored-by: skiffer-git <44203734@qq.com> Co-authored-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> Co-authored-by: withchao <993506633@qq.com>
7 months ago
func NewConversationLocalCache(client rpcclient.ConversationRpcClient, localCache *config.LocalCache, cli redis.UniversalClient) *ConversationLocalCache {
lc := localCache.Conversation
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
log.ZDebug(context.Background(), "ConversationLocalCache", "topic", lc.Topic, "slotNum", lc.SlotNum, "slotSize", lc.SlotSize, "enable", lc.Enable())
x := &ConversationLocalCache{
client: client,
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
local: localcache.New[[]byte](
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
localcache.WithLocalSlotNum(lc.SlotNum),
localcache.WithLocalSlotSize(lc.SlotSize),
localcache.WithLinkSlotNum(lc.SlotNum),
localcache.WithLocalSuccessTTL(lc.Success()),
localcache.WithLocalFailedTTL(lc.Failed()),
),
}
if lc.Enable() {
go subscriberRedisDeleteCache(context.Background(), cli, lc.Topic, x.local.DelLocal)
}
return x
}
type ConversationLocalCache struct {
client rpcclient.ConversationRpcClient
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
local localcache.Cache[[]byte]
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}
func (c *ConversationLocalCache) GetConversationIDs(ctx context.Context, ownerUserID string) (val []string, err error) {
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
resp, err := c.getConversationIDs(ctx, ownerUserID)
if err != nil {
return nil, err
}
return resp.ConversationIDs, nil
}
func (c *ConversationLocalCache) getConversationIDs(ctx context.Context, ownerUserID string) (val *pbconversation.GetConversationIDsResp, err error) {
log.ZDebug(ctx, "ConversationLocalCache getConversationIDs req", "ownerUserID", ownerUserID)
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
defer func() {
if err == nil {
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
log.ZDebug(ctx, "ConversationLocalCache getConversationIDs return", "ownerUserID", ownerUserID, "value", val)
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
} else {
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
log.ZError(ctx, "ConversationLocalCache getConversationIDs return", err, "ownerUserID", ownerUserID)
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}
}()
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
var cache cacheProto[pbconversation.GetConversationIDsResp]
return cache.Unmarshal(c.local.Get(ctx, cachekey.GetConversationIDsKey(ownerUserID), func(ctx context.Context) ([]byte, error) {
log.ZDebug(ctx, "ConversationLocalCache getConversationIDs rpc", "ownerUserID", ownerUserID)
return cache.Marshal(c.client.Client.GetConversationIDs(ctx, &pbconversation.GetConversationIDsReq{UserID: ownerUserID}))
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}))
}
func (c *ConversationLocalCache) GetConversation(ctx context.Context, userID, conversationID string) (val *pbconversation.Conversation, err error) {
log.ZDebug(ctx, "ConversationLocalCache GetConversation req", "userID", userID, "conversationID", conversationID)
defer func() {
if err == nil {
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
log.ZDebug(ctx, "ConversationLocalCache GetConversation return", "userID", userID, "conversationID", conversationID, "value", val)
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
} else {
merge: merge main code into js branch. (#2648) * feat: update group notification when set to null. (#2590) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * feat: update group notification when set to null. * update log standard. * feat: add long time push msg in prometheus (#2584) * feat: add long time push msg in prometheus * fix: log print * fix: go mod * fix: log msg * fix: log init * feat: push msg * feat: go mod ,remove cgo package * feat: remove error log * feat: test dummy push * feat:redis pool config * feat: push to kafka log * feat: supports getting messages based on session ID and seq (#2582) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * feat: implement request batch count limit. (#2591) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * fix: getting messages based on session ID and seq (#2595) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq * feat: join group notification and get seq --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * feat: avoid pulling messages from sessions with a large number of max seq values of 0 (#2602) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq * feat: join group notification and get seq * feat: avoid pulling messages from sessions with a large number of max seq values of 0 --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * refactor: improve db structure in `storage/controller` (#2604) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * refactor: improve db structure in `storage/controller` * feat: implement offline push using kafka (#2600) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * feat: implement offline push. * feat: implement batch Push spilt * update go mod * feat: implement kafka producer and consumer. * update format, * add PushMQ log. * feat: update Handler logic. * update MQ logic. * update * update * fix: update OfflinePushConsumerHandler. * feat: API supports gzip (#2609) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq * feat: join group notification and get seq * feat: avoid pulling messages from sessions with a large number of max seq values of 0 * feat: API supports gzip --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * Fix err (#2608) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * feat: add rocksTimeout * feat: wrap logs * feat: add logs * feat: listen config * feat: enable listen TIME_WAIT port * feat: add logs * feat: cache batch * chore: enable fullUserCache * feat: push rpc num * feat: push err * feat: with operationID * feat: sleep * feat: change 1s * feat: change log * feat: implement Getbatch in rpcCache. * feat: print getOnline cost * feat: change log * feat: change kafka and push config * feat: del interface * feat: fix err * feat: change config * feat: go mod * feat: change config * feat: change config * feat: add sleep in push * feat: warn logs * feat: logs * feat: logs * feat: change port * feat: start config * feat: remove port reuse * feat: prometheus config * feat: prometheus config * feat: prometheus config * feat: add long time send msg to grafana * feat: init * feat: init * feat: implement offline push. * feat: batch get user online * feat: implement batch Push spilt * update go mod * Revert "feat: change port" This reverts commit 06d5e944 * feat: change port * feat: change config * feat: implement kafka producer and consumer. * update format, * add PushMQ log. * feat: get all online users and init push * feat: lock in online cache * feat: config * fix: init online status * fix: add logs * fix: userIDs * fix: add logs * feat: update Handler logic. * update MQ logic. * update * update * fix: method name * fix: update OfflinePushConsumerHandler. * fix: prommetrics * fix: add logs * fix: ctx * fix: log * fix: config * feat: change port * fix: atomic online cache status --------- Co-authored-by: Monet Lee <monet_lee@163.com> * feature: add GetConversationsHasReadAndMaxSeq interface to the WebSocket API. (#2611) * fix: lru lock (#2613) * fix: lru lock * fix: lru lock * fix: lru lock * fix: nil pointer error on close (#2618) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq * feat: join group notification and get seq * feat: avoid pulling messages from sessions with a large number of max seq values of 0 * feat: API supports gzip * go.mod * fix: nil pointer error on close --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * feat: create group can push notification (#2617) * fix: blockage caused by listen error (#2620) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq * feat: join group notification and get seq * feat: avoid pulling messages from sessions with a large number of max seq values of 0 * feat: API supports gzip * go.mod * fix: nil pointer error on close * fix: listen error --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * fix: go.mod (#2621) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq * feat: join group notification and get seq * feat: avoid pulling messages from sessions with a large number of max seq values of 0 * feat: API supports gzip * go.mod * fix: nil pointer error on close * fix: listen error * fix: listen error * update go.mod --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * feat: improve searchMsg implement. (#2614) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * remove unused script. * feat: improve searchMsg implement. * update mongo config. * Fix lock (#2622) * fix:log * fix: lock * fix: update setGroupInfoEX field name. (#2625) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * fix: update setGroupInfoEX field name. * fix: update setGroupInfoEX field name (#2626) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * fix: update setGroupInfoEX field name. * fix: update setGroupInfoEX field name * feat: msg gateway add log (#2631) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq * feat: join group notification and get seq * feat: avoid pulling messages from sessions with a large number of max seq values of 0 * feat: API supports gzip * go.mod * fix: nil pointer error on close * fix: listen error * fix: listen error * update go.mod * feat: add log --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * fix: update setGroupInfoEx func name and field. (#2634) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * fix: update setGroupInfoEx func name and field. * refactor: update groupinfoEx field. * refactor: update database name in mongodb.yml * add groupName Condition * fix: fix setConversations req fill. (#2645) * refactor: refactor workflows contents. * add tool workflows. * update field. * fix: remove chat error. * Fix err. * fix error. * remove cn comment. * update workflows files. * update infra config. * move workflows. * feat: update bot. * fix: solve uncorrect outdated msg get. * update get docIDs logic. * update * update skip logic. * fix * update. * fix: delay deleteObject func. * remove unused content. * update log type. * feat: implement request batch count limit. * update * update * fix: fix setConversations req fill. * fix: GetMsgBySeqs boundary issues (#2647) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * update gomake version * update gomake version * fix: seq conversion bug * fix: redis pipe exec * fix: ImportFriends * fix: A large number of logs keysAndValues ​​length is not even * feat: mark read aggregate write * feat: online status supports redis cluster * feat: online status supports redis cluster * feat: online status supports redis cluster * merge * merge * read seq is written to mongo * read seq is written to mongo * fix: invitation to join group notification * fix: friend op_user_id * feat: optimizing asynchronous context * feat: optimizing memamq size * feat: add GetSeqMessage * feat: GroupApplicationAgreeMemberEnterNotification * feat: GroupApplicationAgreeMemberEnterNotification * feat: go.mod * feat: go.mod * feat: join group notification and get seq * feat: join group notification and get seq * feat: avoid pulling messages from sessions with a large number of max seq values of 0 * feat: API supports gzip * go.mod * fix: nil pointer error on close * fix: listen error * fix: listen error * update go.mod * feat: add log * fix: token parse token value * fix: GetMsgBySeqs boundary issues --------- Co-authored-by: withchao <withchao@users.noreply.github.com> * fix: the attribute version is obsolete, remove it (#2644) * refactor: update Userregister request field. (#2650) --------- Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: icey-yu <119291641+icey-yu@users.noreply.github.com> Co-authored-by: chao <48119764+withchao@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: 蔡相跃 <caixiangyue007@gmail.com>
2 months ago
log.ZWarn(ctx, "ConversationLocalCache GetConversation return", err, "userID", userID, "conversationID", conversationID)
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}
}()
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
var cache cacheProto[pbconversation.Conversation]
return cache.Unmarshal(c.local.Get(ctx, cachekey.GetConversationKey(userID, conversationID), func(ctx context.Context) ([]byte, error) {
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
log.ZDebug(ctx, "ConversationLocalCache GetConversation rpc", "userID", userID, "conversationID", conversationID)
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
return cache.Marshal(c.client.GetConversation(ctx, userID, conversationID))
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}))
}
func (c *ConversationLocalCache) GetSingleConversationRecvMsgOpt(ctx context.Context, userID, conversationID string) (int32, error) {
conv, err := c.GetConversation(ctx, userID, conversationID)
if err != nil {
return 0, err
}
return conv.RecvMsgOpt, nil
}
func (c *ConversationLocalCache) GetConversations(ctx context.Context, ownerUserID string, conversationIDs []string) ([]*pbconversation.Conversation, error) {
var (
conversations = make([]*pbconversation.Conversation, 0, len(conversationIDs))
conversationsChan = make(chan *pbconversation.Conversation, len(conversationIDs))
)
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(conversationWorkerCount)
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
for _, conversationID := range conversationIDs {
conversationID := conversationID
g.Go(func() error {
conversation, err := c.GetConversation(ctx, ownerUserID, conversationID)
if err != nil {
if errs.ErrRecordNotFound.Is(err) {
return nil
}
return err
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}
conversationsChan <- conversation
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
close(conversationsChan)
for conversation := range conversationsChan {
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
conversations = append(conversations, conversation)
}
return conversations, nil
}
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
func (c *ConversationLocalCache) getConversationNotReceiveMessageUserIDs(ctx context.Context, conversationID string) (val *pbconversation.GetConversationNotReceiveMessageUserIDsResp, err error) {
log.ZDebug(ctx, "ConversationLocalCache getConversationNotReceiveMessageUserIDs req", "conversationID", conversationID)
defer func() {
if err == nil {
log.ZDebug(ctx, "ConversationLocalCache getConversationNotReceiveMessageUserIDs return", "conversationID", conversationID, "value", val)
} else {
log.ZError(ctx, "ConversationLocalCache getConversationNotReceiveMessageUserIDs return", err, "conversationID", conversationID)
}
}()
var cache cacheProto[pbconversation.GetConversationNotReceiveMessageUserIDsResp]
return cache.Unmarshal(c.local.Get(ctx, cachekey.GetConversationNotReceiveMessageUserIDsKey(conversationID), func(ctx context.Context) ([]byte, error) {
log.ZDebug(ctx, "ConversationLocalCache getConversationNotReceiveMessageUserIDs rpc", "conversationID", conversationID)
return cache.Marshal(c.client.Client.GetConversationNotReceiveMessageUserIDs(ctx, &pbconversation.GetConversationNotReceiveMessageUserIDsReq{ConversationID: conversationID}))
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}))
}
func (c *ConversationLocalCache) GetConversationNotReceiveMessageUserIDs(ctx context.Context, conversationID string) ([]string, error) {
res, err := c.getConversationNotReceiveMessageUserIDs(ctx, conversationID)
if err != nil {
return nil, err
}
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
return res.UserIDs, nil
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}
func (c *ConversationLocalCache) GetConversationNotReceiveMessageUserIDMap(ctx context.Context, conversationID string) (map[string]struct{}, error) {
res, err := c.getConversationNotReceiveMessageUserIDs(ctx, conversationID)
if err != nil {
return nil, err
}
fix: the local cache obtained can be modified (#2473) * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * cicd: robot automated Change * fix: component * fix: getConversationInfo * feat: cron task * feat: cron task * feat: cron task * feat: cron task * feat: cron task * fix: minio config url recognition error * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * new mongo * friend incr sync * friend incr sync * friend incr sync * friend incr sync * friend incr sync * mage * optimization version log * optimization version log * sync * sync * sync * group sync * sync option * sync option * refactor: replace `friend` package with `realtion`. * refactor: update lastest commit to relation. * sync option * sync option * sync option * sync * sync * go.mod * seq * update: go mod * refactor: change incremental to full * feat: get full friend user ids * feat: api and config * seq * group version * merge * seq * seq * seq * fix: sort by id avoid unstable sort friends. * group * group * group * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * fix: sort by id avoid unstable sort friends. * user version * seq * seq * seq user * user online * implement minio expire delete. * user online * config * fix * fix * implement minio expire delete logic. * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * online cache * feat: implement scheduled delete outdated object in minio. * update gomake version * update gomake version * implement FindExpires pagination. * remove unnesseary incr. * fix uncorrect args call. * online push * online push * online push * resolving conflicts * resolving conflicts * test * api prommetrics * api prommetrics * api prommetrics * api prommetrics * api prommetrics * rpc prommetrics * rpc prommetrics * online status * online status * online status * online status * sub * conversation version incremental * merge seq * merge online * merge online * merge online * merge seq * GetOwnerConversation * fix: change incremental syncer router name. * rockscache batch get * rockscache seq batch get * fix: GetMsgDocModelByIndex bug * update go.mod * update go.mod * merge * feat: prometheus * feat: prometheus * group member sort * sub * sub * fix: seq conversion bug * fix: redis pipe exec * sort version * sort version * sort version * remove old version online subscription * remove old version online subscription * version log index * version log index * batch push * batch push * seq void filling * fix: batchGetMaxSeq * fix: batchGetMaxSeq * cache db error log * 111 * fix bug * fix: ImportFriends * add online cache * add some logs * add some logs * fix: onlineUserIDs * add logs * test * test * test * test * add log * feat: solve the problem that modifying the cached data affects other * feat: solve the problem that modifying the cached data affects other * feat: search messages to filter out notifications --------- Co-authored-by: withchao <withchao@users.noreply.github.com> Co-authored-by: Monet Lee <monet_lee@163.com> Co-authored-by: OpenIM-Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: icey-yu <1186114839@qq.com>
4 months ago
return datautil.SliceSet(res.UserIDs), nil
feat: add local cache for high frequency reads (#2036) * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * feat: msg local cache * fix: mongo * fix: mongo * fix: mongo * openim.yaml * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * localcache * local cache * local cache * local cache * local cache * fix: GroupApplicationAcceptedNotification * fix: GroupApplicationAcceptedNotification * fix: NotificationUserInfoUpdate * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: cache add single-flight and timing-wheel. * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * feat: msg rpc local cache * refactor: refactor the code of push and optimization. * cicd: robot automated Change * refactor: rename cache. * merge * fix: refactor project dir avoid import cycle. * update tools * merge * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * feat: conversation FindRecvMsgNotNotifyUserIDs * merge * merge the latest main --------- Co-authored-by: Gordon <46924906+FGadvancer@users.noreply.github.com> Co-authored-by: withchao <withchao@users.noreply.github.com>
8 months ago
}