feat: set opneim's bash logs

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>
pull/825/head
Xinwei Xiong(cubxxw-openim) 2 years ago
parent 5dd59a51d9
commit 995e68dcae
No known key found for this signature in database
GPG Key ID: 1BAD6F395338EFDE

@ -633,7 +633,7 @@ active_sync_enabled = true
#################################### AWS ###########################
[aws]
# Enter a comma-separated list of allowed AWS authentication providers.
# Options are: default (AWS SDK Default), keys (Access && secret key), credentials (Credentials field), ec2_openim_role (EC2 OpenIM Role)
# Options are: default (AWS SDK Default), keys (Access && secret key), credentials (Credentials field), ec2_iam_role (EC2 OpenIM Role)
allowed_auth_providers = default,keys,credentials
# Allow AWS users to assume a role using temporary security credentials.

@ -48,7 +48,7 @@ jobs:
strategy:
matrix:
go_version: ["1.18","1.19","1.20"]
go_version: ["1.18","1.19","1.20","1.21"]
os: [ubuntu-latest]
steps:
@ -159,7 +159,7 @@ jobs:
name: openim
strategy:
matrix:
go_version: ["1.20"]
go_version: ["1.21"]
os: ["ubuntu-latest"]
steps:
- name: Set up Go ${{ matrix.go_version }}

@ -0,0 +1,47 @@
## OpenIM Logging System: Design and Usage
**PATH:** `scripts/lib/logging.sh`
### Introduction
OpenIM, an intricate project, requires a robust logging mechanism to diagnose issues, maintain system health, and provide insights. A custom-built logging system embedded within OpenIM ensures consistent and structured logs. Let's delve into the design of this logging system and understand its various functions and their usage scenarios.
### Design Overview
1. **Initialization**: The system begins by determining the verbosity level through the `OPENIM_VERBOSE` variable. If it's not set, a default value of 5 is assigned. This verbosity level dictates the depth of the log details.
2. **Log File Setup**: Logs are stored in the directory specified by `OPENIM_OUTPUT`. If this variable isn't explicitly set, it defaults to the `_output` directory relative to the script location. Each log file is named based on the date to facilitate easy identification.
3. **Logging Function**: The `echo_log()` function plays a pivotal role by writing messages to both the console (stdout) and the log file.
4. **Logging to a file**: The `echo_log()` function writes to the log file by appending the message to the file. It also adds a timestamp to the message. path: `_output/logs/*`, Enable logging by default. Set to false to disable. If you wish to turn off output to log files set `ENABLE_LOGGING=flase`.
### Key Functions & Their Usages
1. **Error Handling**:
- `openim::log::errexit()`: Activated when a command exits with an error. It prints a call tree showing the sequence of functions leading to the error and then calls `openim::log::error_exit()` with relevant information.
- `openim::log::install_errexit()`: Sets up the trap for catching errors and ensures that the error handler (`errexit`) gets propagated to various script constructs like functions, expansions, and subshells.
2. **Logging Levels**:
- `openim::log::error()`: Logs error messages with a timestamp. The log message starts with '!!!' to indicate its severity.
- `openim::log::info()`: Provides informational messages. The display of these messages is governed by the verbosity level (`OPENIM_VERBOSE`).
- `openim::log::progress()`: Designed for logging progress messages or creating progress bars.
- `openim::log::status()`: Logs status messages with a timestamp, prefixing each entry with '+++' for easy identification.
- `openim::log::success()`: Highlights successful operations with a bright green prefix. It's ideal for visually signifying operations that completed successfully.
3. **Exit and Stack Trace**:
- `openim::log::error_exit()`: Logs an error message, dumps the call stack, and exits the script with a specified exit code.
- `openim::log::stack()`: Prints out a stack trace, showing the call hierarchy leading to the point where this function was invoked.
4. **Usage Information**:
- `openim::log::usage() & openim::log::usage_from_stdin()`: Both functions provide a mechanism to display usage instructions. The former accepts arguments directly, while the latter reads them from stdin.
5. **Test Function**:
- `openim::log::test_log()`: This function is a test suite to verify that all logging functions are operating as expected.
### Usage Scenario
Imagine a situation where an OpenIM operation fails, and you need to ascertain the cause. With the logging system in place, you can:
- Check the log file for the specific day to find error messages with the '!!!' prefix.
- View the call tree and stack trace to trace back the sequence of operations leading to the failure.
- Use the verbosity level to filter out unnecessary details and focus on the crux of the issue.
This systematic and structured approach greatly simplifies the debugging process, making system maintenance more efficient.
### Conclusion
OpenIM's logging system is a testament to the importance of structured and detailed logging in complex projects. By using this logging mechanism, developers and system administrators can streamline troubleshooting and ensure the seamless operation of the OpenIM project.

@ -68,6 +68,13 @@ The purpose of having a structured scripts directory like this is to make the op
Each directory and script in the structure should be understood as a part of a larger whole. All scripts work together to ensure the smooth operation and maintenance of the OpenIM Server.
## log directory
**PATH:** `scripts/lib/logging.sh`
+ [log details](../docs/conversions/bash_log.md)
## Supported platforms
- Linux x86_64 (linux_amd64) : 64-bit Linux for most desktop and server systems.
@ -95,6 +102,10 @@ Steps to run demo:
make demo
```
More about `make` read:
+ [makefile](../docs/conversions/go_code.md)
Instructions for producing the demo movie:
```bash

@ -14,12 +14,22 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#Include shell font styles and some basic information
SCRIPTS_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# This script runs `make build` command.
# The command compiles all Makefile configs.
# Args:
# WHAT: Directory names to build. If any of these directories has a 'main'
# package, the build will produce executable files under $(OUT_DIR)/go/bin.
# If not specified, "everything" will be built.
# Usage: `hack/build-go.sh`.
# Example: `hack/build-go.sh WHAT=cmd/kubelet`.
set -o errexit
set -o nounset
set -o pipefail
OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
source "${OPENIM_ROOT}/scripts/lib/init.sh"
#Include shell font styles and some basic information
source $SCRIPTS_ROOT/path_info.sh
source $SCRIPTS_ROOT/lib/init.sh
bin_dir="$BIN_DIR"

@ -13,19 +13,21 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#Include shell font styles and some basic information
SCRIPTS_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
# This script is check openim service is running normally
#
# Usage: `scripts/check_all.sh`.
# Encapsulated as: `make check`.
#Include shell font styles and some basic information
source $SCRIPTS_ROOT/lib/init.sh
source $SCRIPTS_ROOT/path_info.sh
set -o errexit
set -o nounset
set -o pipefail
OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
source "${OPENIM_ROOT}/scripts/lib/init.sh"
cd $SCRIPTS_ROOT
source "${OPENIM_ROOT}/scripts/path_info.sh"
echo -e "${YELLOW_PREFIX}=======>SCRIPTS_ROOT=$SCRIPTS_ROOT${COLOR_SUFFIX}"
echo -e "${YELLOW_PREFIX}=======>OPENIM_ROOT=$OPENIM_ROOT${COLOR_SUFFIX}"
echo -e "${YELLOW_PREFIX}=======>pwd=$PWD${COLOR_SUFFIX}"
OPENIM_VERBOSE=4
service_port_name=(
openImWsPort

@ -1,8 +1,17 @@
#!/usr/bin/env bash
# Copyright 2020 Lingfei Kong <colin404@foxmail.com>. All rights reserved.
# Use of this source code is governed by a MIT style
# license that can be found in the LICENSE file.
# Copyright © 2023 OpenIM. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# 本脚本功能:根据 scripts/environment.sh 配置,生成 OPENIM 组件 YAML 配置文件。
# 示例:./scripts/genconfig.sh scripts/install/environment.sh scripts/template/openim_config.yaml

@ -1,8 +1,17 @@
#!/usr/bin/env bash
# Copyright 2020 Lingfei Kong <colin404@foxmail.com>. All rights reserved.
# Use of this source code is governed by a MIT style
# license that can be found in the LICENSE file.
# Copyright © 2023 OpenIM. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Common utilities, variables and checks for all build scripts.

@ -1,11 +1,23 @@
#!/usr/bin/env bash
# Copyright © 2023 OpenIM. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/../..
# This script will install the dependencies required for openim
# Setting bash variables as readonly
readonly DATA_DIR="/path/to/data/dir" # replace with the actual path
readonly USER="your_user"
readonly PASSWORD="your_password"
OPENIM_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)"
[[ -z ${COMMON_SOURCED} ]] && source ${OPENIM_ROOT}/scripts/install/common.sh
# Start MySQL service
docker run -d \

@ -1,10 +1,18 @@
#!/usr/bin/env bash
# Copyright © 2023 OpenIM. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Copyright 2020 Lingfei Kong <colin404@foxmail.com>. All rights reserved.
# Use of this source code is governed by a MIT style
# license that can be found in the LICENSE file.
# OPENIM 项目源码根目录
OPENIM_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)"
# 生成文件存放目录
@ -15,10 +23,10 @@ readonly API_URL=${API_URL:-http://127.0.0.1:10002/object/}
readonly DATA_DIR=${DATA_DIR:-${OPENIM_ROOT}}
# 设置统一的用户名,方便记忆
readonly USER=${USER:-'root'}
readonly USER=${USER:-'root'} # Setting a username
# 设置统一的密码,方便记忆
readonly PASSWORD=${PASSWORD:-'openIM123'}
readonly PASSWORD=${PASSWORD:-'openIM123'} # Setting a password
# Linux系统 going 用户
readonly LINUX_USERNAME=${LINUX_USERNAME:-going}

@ -1,9 +1,17 @@
#!/bin/bash
# Copyright 2020 Lingfei Kong <colin404@foxmail.com>. All rights reserved.
# Use of this source code is governed by a MIT style
# license that can be found in the LICENSE file.
#!/usr/bin/env bash
# Copyright © 2023 OpenIM. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The root of the build/dist directory
OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/../..

@ -16,16 +16,24 @@
# Controls verbosity of the script output and logging.
OPENIM_VERBOSE="${OPENIM_VERBOSE:-5}"
if [[ ! -v OPENIM_VERBOSE ]]; then
openim::log::info "OPENIM_VERBOSE is not set; defaulting to \"5\""
# Enable logging by default. Set to false to disable.
ENABLE_LOGGING=true
# If OPENIM_OUTPUT is not set, set it to the default value
if [[ ! -v OPENIM_OUTPUT ]]; then
OPENIM_OUTPUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../_output" && pwd -P)"
OPENIM_VERBOSE="${OPENIM_OUTPUT}/logs"
fi
log_file="${OPENIM_OUTPUT_LOGS}/openim_$(date '+%Y%m%d').log"
# Set the log file path
log_file="${OPENIM_OUTPUT}/logs/openim_$(date '+%Y%m%d').log"
# Define the logging function
function echo_log() {
echo_log -e "$@" | tee -a "${log_file}"
if $ENABLE_LOGGING; then
echo -e "$@" | tee -a "${log_file}"
else
echo -e "$@"
fi
}
# MAX_LOG_SIZE=10485760 # 10MB
@ -74,7 +82,7 @@ openim::log::stack() {
local stack_skip=${1:-0}
stack_skip=$((stack_skip + 1))
if [[ ${#FUNCNAME[@]} -gt ${stack_skip} ]]; then
echo_log_log "Call stack:" >&2
echo_log "Call stack:" >&2
local i
for ((i=1 ; i <= ${#FUNCNAME[@]} - stack_skip ; i++))
do
@ -82,7 +90,7 @@ openim::log::stack() {
local source_file=${BASH_SOURCE[${frame_no}]}
local source_lineno=${BASH_LINENO[$((frame_no - 1))]}
local funcname=${FUNCNAME[${frame_no}]}
echo_log_log " ${i}: ${source_file}:${source_lineno} ${funcname}(...)" >&2
echo_log " ${i}: ${source_file}:${source_lineno} ${funcname}(...)" >&2
done
fi
}
@ -194,3 +202,15 @@ openim::log::success()
fi
echo_log -e "${BRIGHT_GREEN_PREFIX}===> [success] <===${COLOR_SUFFIX}\n=> " "$@"
}
function openim::log::test_log() {
echo_log "test log"
openim::log::error "openim::log::error"
openim::log::info "openim::log::info"
openim::log::progress "openim::log::progress"
openim::log::status "openim::log::status"
openim::log::success "openim::log::success"
openim::log::error_exit "openim::log::error_exit"
}
# openim::log::test_log

@ -457,11 +457,6 @@ EOF
cp -R "${OPENIM_ROOT}/scripts/release" "${release_stage}/"
cat <<EOF > "${release_stage}/release/get-openim-binaries.sh"
#!/usr/bin/env bash
# Copyright 2020 Lingfei Kong <colin404@foxmail.com>. All rights reserved.
# Use of this source code is governed by a MIT style
# license that can be found in the LICENSE file.
# This file download openim client and server binaries from tencent cos bucket.
os=linux arch=amd64 version=${OPENIM_GIT_VERSION} && wget https://${BUCKET}.cos.${REGION}.myqcloud.com/${COS_RELEASE_DIR}/\$version/{openim-client-\$os-\$arch.tar.gz,openim-server-\$os-\$arch.tar.gz}

@ -0,0 +1,12 @@
#!/usr/bin/env bash
# This script lists all of the [Feature:.+] tests in our e2e suite.
#
# Usage: `scripts/list-feature-tests.sh`.
set -o errexit
set -o nounset
set -o pipefail
OPENIM_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
grep "\[Feature:\w+\]" "${OPENIM_ROOT}"/test/e2e/**/*.go -Eoh | LC_ALL=C sort -u

@ -100,7 +100,7 @@ endif
# The OS must be linux when building docker images
# PLATFORMS ?= linux_amd64 linux_arm64
# The OS can be linux/windows/darwin when building binaries
PLATFORMS ?= linux_s390x linux_mips64 linux_mips64le darwin_amd64 windows_amd64 linux_amd64 linux_arm64 linux_ppc64le
PLATFORMS ?= linux_s390x linux_mips64 linux_mips64le darwin_amd64 windows_amd64 linux_amd64 linux_arm64 linux_ppc64le # wasip1_wasm
# only support linux
GOOS=linux

@ -23,10 +23,6 @@ source $SCRIPTS_ROOT/path_info.sh
cd $SCRIPTS_ROOT
echo -e "${YELLOW_PREFIX}=======>SCRIPTS_ROOT=$SCRIPTS_ROOT${COLOR_SUFFIX}"
echo -e "${YELLOW_PREFIX}=======>OPENIM_ROOT=$OPENIM_ROOT${COLOR_SUFFIX}"
echo -e "${YELLOW_PREFIX}=======>pwd=$PWD${COLOR_SUFFIX}"
bin_dir="$BIN_DIR"
logs_dir="$OPENIM_ROOT/logs"

@ -1,3 +1,17 @@
# Copyright © 2023 OpenIM. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
object:
enable: "${OBJECT_ENABLE}"

@ -1,8 +1,17 @@
#!/bin/bash
# Copyright 2020 Lingfei Kong <colin404@foxmail.com>. All rights reserved.
# Use of this source code is governed by a MIT style
# license that can be found in the LICENSE file.
#!/usr/bin/env bash
# Copyright © 2023 OpenIM. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
: << EOF
The API performance test script automatically executes wrk commands, collects data, analyzes it, and calls gnuplot to plot it

Loading…
Cancel
Save