Merge pull request #1820 from openimsdk/feat/openim-auto-releasse
feat(release-v3.5): Provide more automation, as well as build capabilitiesrelease-v3.5 v3.5.1-alpha.2
Xinwei Xiong
2 years ago
committed by
GitHub
commit
093b3f640c
@ -0,0 +1,45 @@
|
|||||||
|
# Copyright © 2023 OpenIM. All rights reserved.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
# OpenIM base image: https://github.com/openim-sigs/openim-base-image
|
||||||
|
|
||||||
|
# Set go mod installation source and proxy
|
||||||
|
|
||||||
|
FROM golang:1.20 AS builder
|
||||||
|
|
||||||
|
ARG GO111MODULE=on
|
||||||
|
ARG GOPROXY=https://goproxy.cn,direct
|
||||||
|
|
||||||
|
WORKDIR /openim/openim-server
|
||||||
|
|
||||||
|
ENV GO111MODULE=$GO111MODULE
|
||||||
|
ENV GOPROXY=$GOPROXY
|
||||||
|
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
RUN make build BINS=openim-rpc-encryption
|
||||||
|
|
||||||
|
RUN cp /openim/openim-server/_output/bin/platforms/$(go env GOOS)/$(go env GOARCH)/openim-rpc-encryption /usr/bin/openim-rpc-encryption
|
||||||
|
|
||||||
|
# FROM ghcr.io/openim-sigs/openim-bash-image:latest
|
||||||
|
FROM ghcr.io/openim-sigs/openim-bash-image:latest
|
||||||
|
|
||||||
|
WORKDIR /openim/openim-server
|
||||||
|
|
||||||
|
COPY --from=builder /usr/bin/openim-rpc-encryption ./bin/openim-rpc-encryption
|
||||||
|
|
||||||
|
ENTRYPOINT ["./bin/openim-rpc-encryption"]
|
||||||
@ -0,0 +1,45 @@
|
|||||||
|
# Copyright © 2023 OpenIM. All rights reserved.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
# OpenIM base image: https://github.com/openim-sigs/openim-base-image
|
||||||
|
|
||||||
|
# Set go mod installation source and proxy
|
||||||
|
|
||||||
|
FROM golang:1.20 AS builder
|
||||||
|
|
||||||
|
ARG GO111MODULE=on
|
||||||
|
ARG GOPROXY=https://goproxy.cn,direct
|
||||||
|
|
||||||
|
WORKDIR /openim/openim-server
|
||||||
|
|
||||||
|
ENV GO111MODULE=$GO111MODULE
|
||||||
|
ENV GOPROXY=$GOPROXY
|
||||||
|
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
RUN make build BINS=openim-rpc-extend-msg
|
||||||
|
|
||||||
|
RUN cp /openim/openim-server/_output/bin/platforms/$(go env GOOS)/$(go env GOARCH)/openim-rpc-extend-msg /usr/bin/openim-rpc-extend-msg
|
||||||
|
|
||||||
|
# FROM ghcr.io/openim-sigs/openim-bash-image:latest
|
||||||
|
FROM ghcr.io/openim-sigs/openim-bash-image:latest
|
||||||
|
|
||||||
|
WORKDIR /openim/openim-server
|
||||||
|
|
||||||
|
COPY --from=builder /usr/bin/openim-rpc-extend-msg ./bin/openim-rpc-extend-msg
|
||||||
|
|
||||||
|
ENTRYPOINT ["./bin/openim-rpc-extend-msg"]
|
||||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,258 @@
|
|||||||
|
# Mac Developer Deployment Guide for OpenIM
|
||||||
|
|
||||||
|
## Introduction
|
||||||
|
|
||||||
|
This guide aims to assist Mac-based developers in contributing effectively to OpenIM. It covers the setup of a development environment tailored for Mac, including the use of GitHub for development workflow and `devcontainer` for a consistent development experience.
|
||||||
|
|
||||||
|
Before contributing to OpenIM through issues and pull requests, make sure you are familiar with GitHub and the [pull request workflow](https://docs.github.com/en/get-started/quickstart/github-flow).
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
### System Requirements
|
||||||
|
|
||||||
|
- macOS (latest stable version recommended)
|
||||||
|
- Internet connection
|
||||||
|
- Administrator access
|
||||||
|
|
||||||
|
### Knowledge Requirements
|
||||||
|
|
||||||
|
- Basic understanding of Git and GitHub
|
||||||
|
- Familiarity with Docker and containerization
|
||||||
|
- Experience with Go programming language
|
||||||
|
|
||||||
|
## Setting up the Development Environment
|
||||||
|
|
||||||
|
### Installing Homebrew
|
||||||
|
|
||||||
|
Homebrew is an essential package manager for macOS. Install it using:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Installing and Configuring Git
|
||||||
|
|
||||||
|
1. Install Git:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
brew install git
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Configure Git with your user details:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git config --global user.name "Your Name"
|
||||||
|
git config --global user.email "your.email@example.com"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Setting Up the Devcontainer
|
||||||
|
|
||||||
|
`Devcontainers` provide a Docker-based isolated development environment.
|
||||||
|
|
||||||
|
Read [README.md](https://github.com/openimsdk/open-im-server/tree/main/.devcontainer) in the `.devcontainer` directory of the project to learn more about the devcontainer.
|
||||||
|
|
||||||
|
To set it up:
|
||||||
|
|
||||||
|
1. Install Docker Desktop for Mac from [Docker Hub](https://docs.docker.com/desktop/install/mac-install/).
|
||||||
|
2. Install Visual Studio Code and the Remote - Containers extension.
|
||||||
|
3. Open the cloned OpenIM repository in VS Code.
|
||||||
|
4. VS Code will prompt to reopen the project in a container. Accept this to set up the environment automatically.
|
||||||
|
|
||||||
|
### Installing Go and Dependencies
|
||||||
|
|
||||||
|
Use Homebrew to install Go:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
brew install go
|
||||||
|
```
|
||||||
|
|
||||||
|
Ensure the version of Go is compatible with the version required by OpenIM (refer to the main documentation for version requirements).
|
||||||
|
|
||||||
|
### Additional Tools
|
||||||
|
|
||||||
|
Install other required tools like Docker, Vagrant, and necessary GNU utils as described in the main documentation.
|
||||||
|
|
||||||
|
## Mac Deployment openim-chat and openim-server
|
||||||
|
|
||||||
|
To integrate the Chinese document into an English document for Linux deployment, we will first translate the content and then adapt it to suit the Linux environment. Here's how the translated and adapted content might look:
|
||||||
|
|
||||||
|
### Ensure a Clean Environment
|
||||||
|
|
||||||
|
- It's recommended to execute in a new directory.
|
||||||
|
- Run `ps -ef | grep openim` to ensure no OpenIM processes are running.
|
||||||
|
- Run `ps -ef | grep chat` to check for absence of chat-related processes.
|
||||||
|
- Execute `docker ps` to verify there are no related containers running.
|
||||||
|
|
||||||
|
### Source Code Deployment
|
||||||
|
|
||||||
|
#### Deploying openim-server
|
||||||
|
|
||||||
|
Source code deployment is slightly more complex because Docker's networking on Linux differs from Mac.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/openimsdk/open-im-server
|
||||||
|
cd open-im-server
|
||||||
|
|
||||||
|
export OPENIM_IP="Your IP" # If it's a cloud server, setting might not be needed
|
||||||
|
make init # Generates configuration files
|
||||||
|
```
|
||||||
|
|
||||||
|
Before deploying openim-server, modify the Kafka logic in the docker-compose.yml file. Replace:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,EXTERNAL://${DOCKER_BRIDGE_GATEWAY:-172.28.0.1}:${KAFKA_PORT:-19094}
|
||||||
|
```
|
||||||
|
|
||||||
|
With:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,EXTERNAL://127.0.0.1:${KAFKA_PORT:-19094}
|
||||||
|
```
|
||||||
|
|
||||||
|
Then start the service:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Before starting the openim-server source, set `config/config.yaml` by replacing all instances of `172.28.0.1` with `127.0.0.1`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
vim config/config.yaml -c "%s/172\.28\.0\.1/127.0.0.1/g" -c "wq"
|
||||||
|
```
|
||||||
|
|
||||||
|
Then start openim-server:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make start
|
||||||
|
```
|
||||||
|
|
||||||
|
To check the startup:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make check
|
||||||
|
```
|
||||||
|
|
||||||
|
<aside>
|
||||||
|
🚧 To avoid mishaps, it's best to wait five minutes before running `make check` again.
|
||||||
|
|
||||||
|
</aside>
|
||||||
|
|
||||||
|
#### Deploying openim-chat
|
||||||
|
|
||||||
|
There are several ways to deploy openim-chat, either by source code or using Docker.
|
||||||
|
|
||||||
|
Navigate back to the parent directory:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ..
|
||||||
|
```
|
||||||
|
|
||||||
|
First, let's look at deploying chat from source:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/openimsdk/chat
|
||||||
|
cd chat
|
||||||
|
make init # Generates configuration files
|
||||||
|
```
|
||||||
|
|
||||||
|
If openim-chat has not deployed MySQL, you will need to deploy it. Note that the official Docker Hub for MySQL does not support architectures like ARM, so you can use the newer version of the open-source edition:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d \
|
||||||
|
--name mysql \
|
||||||
|
-p 13306:3306 \
|
||||||
|
-p 3306:33060 \
|
||||||
|
-v "$(pwd)/components/mysql/data:/var/lib/mysql" \
|
||||||
|
-v "/etc/localtime:/etc/localtime" \
|
||||||
|
-e MYSQL_ROOT_PASSWORD="openIM123" \
|
||||||
|
--restart always \
|
||||||
|
mariadb:10.6
|
||||||
|
```
|
||||||
|
|
||||||
|
Before starting the source code of openim-chat, set `config/config.yaml` by replacing all instances of `172.28.0.1` with `127.0.0.1`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
vim config/config.yaml -c "%s/172\.28\.0\.1/127.0.0.1/g" -c "wq"
|
||||||
|
```
|
||||||
|
|
||||||
|
Then start openim-chat from source:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make start
|
||||||
|
```
|
||||||
|
|
||||||
|
To check, ensure the following four processes start successfully:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make check
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker Deployment
|
||||||
|
|
||||||
|
Refer to https://github.com/openimsdk/openim-docker for Docker deployment instructions, which can be followed similarly on Linux.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/openimsdk/openim-docker
|
||||||
|
cd openim-docker
|
||||||
|
export OPENIM_IP="Your IP"
|
||||||
|
make init
|
||||||
|
docker compose up -d
|
||||||
|
docker compose logs -f openim-server
|
||||||
|
docker compose logs -f openim-chat
|
||||||
|
```
|
||||||
|
|
||||||
|
## GitHub Development Workflow
|
||||||
|
|
||||||
|
### Creating a New Branch
|
||||||
|
|
||||||
|
For new features or fixes, create a new branch:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git checkout -b feat/your-feature-name
|
||||||
|
```
|
||||||
|
|
||||||
|
### Making Changes and Committing
|
||||||
|
|
||||||
|
1. Make your changes in the code.
|
||||||
|
2. Stage your changes:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git add .
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Commit with a meaningful message:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git commit -m "Add a brief description of your changes"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pushing Changes and Creating Pull Requests
|
||||||
|
|
||||||
|
1. Push your branch to GitHub:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git push origin feat/your-feature-name
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Go to your fork on GitHub and create a pull request to the main OpenIM repository.
|
||||||
|
|
||||||
|
### Keeping Your Fork Updated
|
||||||
|
|
||||||
|
Regularly sync your fork with the main repository:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git fetch upstream
|
||||||
|
git checkout main
|
||||||
|
git rebase upstream/main
|
||||||
|
```
|
||||||
|
|
||||||
|
More read: [https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md](https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md)
|
||||||
|
|
||||||
|
## Testing and Quality Assurance
|
||||||
|
|
||||||
|
Run tests as described in the OpenIM documentation to ensure your changes do not break existing functionality.
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
This guide provides a comprehensive overview for Mac developers to set up and contribute to OpenIM. By following these steps, you can ensure a smooth and efficient development experience. Happy coding!
|
||||||
@ -0,0 +1,251 @@
|
|||||||
|
# OpenIM Release Automation Design Document
|
||||||
|
|
||||||
|
This document outlines the automation process for releasing OpenIM. You can use the `make release` command for automated publishing. We will discuss how to use the `make release` command and Github Actions CICD separately, while also providing insight into the design principles involved.
|
||||||
|
|
||||||
|
## Github Actions Automation
|
||||||
|
|
||||||
|
In our CICD pipeline, we have implemented logic for automating the release process using the goreleaser tool. To achieve this, follow these steps on your local machine or server:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/openimsdk/open-im-server
|
||||||
|
cd open-im-server
|
||||||
|
git tag -a v3.6.0 -s -m "release: xxx"
|
||||||
|
# For pre-release versions: git tag -a v3.6.0-rc.0 -s -m "pre-release: xxx"
|
||||||
|
git push origin v3.6.0
|
||||||
|
```
|
||||||
|
|
||||||
|
The remaining tasks are handled by automated processes:
|
||||||
|
|
||||||
|
+ Automatically complete the release publication on Github
|
||||||
|
+ Automatically build the `v3.6.0` version image and push it to aliyun, dockerhub, and github
|
||||||
|
|
||||||
|
Through these automated steps, we achieve rapid and efficient OpenIM version releases, simplifying the release process and enhancing productivity.
|
||||||
|
|
||||||
|
|
||||||
|
Certainly, here is the continuation of the document in English:
|
||||||
|
|
||||||
|
## Local Make Release Design
|
||||||
|
|
||||||
|
There are two primary scenarios for local usage:
|
||||||
|
|
||||||
|
+ Advanced compilation and release, manually executed locally
|
||||||
|
+ Quick compilation verification and version release, manually executed locally
|
||||||
|
|
||||||
|
**These two scenarios can also be combined, for example, by tagging locally and then releasing:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add .
|
||||||
|
git commit -a -s -m "release(v3.6.0): ......"
|
||||||
|
git tag v3.6.0
|
||||||
|
git release
|
||||||
|
git push origin main
|
||||||
|
```
|
||||||
|
|
||||||
|
In a local environment, you can use the `make release` command to complete the release process. The main implementation logic can be found in the `/data/workspaces/open-im-server/scripts/lib/release.sh` file. First, let's explore its usage through the help information.
|
||||||
|
|
||||||
|
### Help Information
|
||||||
|
|
||||||
|
To view the help information, execute the following command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ ./scripts/release.sh --help
|
||||||
|
Usage: release.sh [options]
|
||||||
|
Options:
|
||||||
|
-h, --help Display this help message
|
||||||
|
-se, --setup-env Execute environment setup
|
||||||
|
-vp, --verify-prereqs Execute prerequisite verification
|
||||||
|
-bc, --build-command Execute build command
|
||||||
|
-bi, --build-image Execute build image (default is not executed)
|
||||||
|
-pt, --package-tarballs Execute tarball packaging
|
||||||
|
-ut, --upload-tarballs Execute tarball upload
|
||||||
|
-gr, --github-release Execute GitHub release
|
||||||
|
-gc, --generate-changelog Execute changelog generation
|
||||||
|
```
|
||||||
|
|
||||||
|
### Default Behavior
|
||||||
|
|
||||||
|
If no options are provided, all operations are executed by default:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# If no options are provided, enable all operations by default
|
||||||
|
if [ "$#" -eq 0 ]; then
|
||||||
|
perform_setup_env=true
|
||||||
|
perform_verify_prereqs=true
|
||||||
|
perform_build_command=true
|
||||||
|
perform_package_tarballs=true
|
||||||
|
perform_upload_tarballs=true
|
||||||
|
perform_github_release=true
|
||||||
|
perform_generate_changelog=true
|
||||||
|
# TODO: Defaultly not enable build_image
|
||||||
|
# perform_build_image=true
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
|
||||||
|
### Environment Variable Setup
|
||||||
|
|
||||||
|
Before starting, you need to set environment variables:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export TENCENT_SECRET_KEY=OZZ****************************
|
||||||
|
export TENCENT_SECRET_ID=AKI****************************
|
||||||
|
```
|
||||||
|
|
||||||
|
### Modifying COS Account and Password
|
||||||
|
|
||||||
|
If you need to change the COS account, password, and bucket information, please modify the following section in the `/data/workspaces/open-im-server/scripts/lib/release.sh` file:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
readonly BUCKET="openim-1306374445"
|
||||||
|
readonly REGION="ap-guangzhou"
|
||||||
|
readonly COS_RELEASE_DIR="openim-release"
|
||||||
|
```
|
||||||
|
|
||||||
|
### GitHub Release Configuration
|
||||||
|
|
||||||
|
If you intend to use the GitHub Release feature, you also need to set the environment variable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export GITHUB_TOKEN="your_github_token"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Modifying GitHub Release Basic Information
|
||||||
|
|
||||||
|
If you need to modify the basic information of GitHub Release, please edit the following section in the `/data/workspaces/open-im-server/scripts/lib/release.sh` file:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# OpenIM GitHub account information
|
||||||
|
readonly OPENIM_GITHUB_ORG=openimsdk
|
||||||
|
readonly OPENIM_GITHUB_REPO=open-im-server
|
||||||
|
```
|
||||||
|
|
||||||
|
This setup allows you to configure and execute the local release process according to your specific needs.
|
||||||
|
|
||||||
|
|
||||||
|
### GitHub Release Versioning Rules
|
||||||
|
|
||||||
|
Firstly, it's important to note that GitHub Releases should primarily be for pre-release versions. However, goreleaser might provide a `prerelease: auto` option, which automatically marks versions with pre-release indicators like `-rc1`, `-beta`, etc., as pre-releases.
|
||||||
|
|
||||||
|
So, if your most recent tag does not have pre-release indicators such as `-rc1` or `-beta`, even if you use `make release` for pre-release versions, goreleaser might still consider them as formal releases.
|
||||||
|
|
||||||
|
To avoid this issue, I have added the `--draft` flag to github-release. This way, all releases are created as drafts.
|
||||||
|
|
||||||
|
## CICD Release Documentation Design
|
||||||
|
|
||||||
|
The release records still require manual composition for GitHub Release. This is different from github-release.
|
||||||
|
|
||||||
|
This approach ensures that all releases are initially created as drafts, allowing you to manually review and edit the release documentation on GitHub. This manual step provides more control and allows you to curate release notes and other information before making them public.
|
||||||
|
|
||||||
|
|
||||||
|
## Makefile Section
|
||||||
|
|
||||||
|
This document aims to elaborate and explain key sections of the OpenIM Release automation design, including the Makefile section and functions within the code. Below, we will provide a detailed explanation of the logic and functions of each section.
|
||||||
|
|
||||||
|
In the project's root directory, the Makefile imports a subdirectory:
|
||||||
|
|
||||||
|
```makefile
|
||||||
|
include scripts/make-rules/release.mk
|
||||||
|
```
|
||||||
|
|
||||||
|
And defines the `release` target as follows:
|
||||||
|
|
||||||
|
```makefile
|
||||||
|
## release: release the project ✨
|
||||||
|
.PHONY: release release: release.verify release.ensure-tag
|
||||||
|
@scripts/release.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### Importing Subdirectory
|
||||||
|
|
||||||
|
At the beginning of the Makefile, the `include scripts/make-rules/release.mk` statement imports the `release.mk` file from the subdirectory. This file contains rules and configurations related to releases to be used in subsequent operations.
|
||||||
|
|
||||||
|
### The `release` Target
|
||||||
|
|
||||||
|
The Makefile defines a target named `release`, which is used to execute the project's release operation. This target is marked as a phony target (`.PHONY`), meaning it doesn't represent an actual file or directory but serves as an identifier for executing a series of actions.
|
||||||
|
|
||||||
|
In the `release` target, two dependency targets are executed first: `release.verify` and `release.ensure-tag`. Afterward, the `scripts/release.sh` script is called to perform the actual release operation.
|
||||||
|
|
||||||
|
## Logic of `release.verify` and `release.ensure-tag`
|
||||||
|
|
||||||
|
```makefile
|
||||||
|
## release.verify: Check if a tool is installed and install it
|
||||||
|
.PHONY: release.verify
|
||||||
|
release.verify: tools.verify.git-chglog tools.verify.github-release tools.verify.coscmd tools.verify.coscli
|
||||||
|
|
||||||
|
## release.ensure-tag: ensure tag
|
||||||
|
.PHONY: release.ensure-tag
|
||||||
|
release.ensure-tag: tools.verify.gsemver
|
||||||
|
@scripts/ensure-tag.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### `release.verify` Target
|
||||||
|
|
||||||
|
The `release.verify` target is used to check and install tools. It depends on four sub-targets: `tools.verify.git-chglog`, `tools.verify.github-release`, `tools.verify.coscmd`, and `tools.verify.coscli`. These sub-targets aim to check if specific tools are installed and attempt to install them if they are not.
|
||||||
|
|
||||||
|
The purpose of this target is to ensure that the necessary tools required for the release process are available so that subsequent operations can be executed smoothly.
|
||||||
|
|
||||||
|
### `release.ensure-tag` Target
|
||||||
|
|
||||||
|
The `release.ensure-tag` target is used to ensure that the project has a version tag. It depends on the sub-target `tools.verify.gsemver`, indicating that it should check if the `gsemver` tool is installed before executing.
|
||||||
|
|
||||||
|
When the `release.ensure-tag` target is executed, it calls the `scripts/ensure-tag.sh` script to ensure that the project has a version tag. Version tags are typically used to identify specific versions of the project for management and release in version control systems.
|
||||||
|
|
||||||
|
## Logic of `release.sh` Script
|
||||||
|
|
||||||
|
```bash
|
||||||
|
openim::golang::setup_env
|
||||||
|
openim::build::verify_prereqs
|
||||||
|
openim::release::verify_prereqs
|
||||||
|
#openim::build::build_image
|
||||||
|
openim::build::build_command
|
||||||
|
openim::release::package_tarballs
|
||||||
|
openim::release::upload_tarballs
|
||||||
|
git push origin ${VERSION}
|
||||||
|
#openim::release::github_release
|
||||||
|
#openim::release::generate_changelog
|
||||||
|
```
|
||||||
|
|
||||||
|
The `release.sh` script is responsible for executing the actual release operations. Below is the logic of this script:
|
||||||
|
|
||||||
|
1. `openim::golang::setup_env`: This function sets up some configurations for the Golang development environment.
|
||||||
|
|
||||||
|
2. `openim::build::verify_prereqs`: This function is used to verify whether the prerequisites for building are met. This includes checking dependencies, environment variables, and more.
|
||||||
|
|
||||||
|
3. `openim::release::verify_prereqs`: Similar to the previous function, this one is used to verify whether the prerequisites for the release are met. It focuses on conditions relevant to the release.
|
||||||
|
|
||||||
|
4. `openim::build::build_command`: This function is responsible for building the project's command, which typically involves compiling the project or performing other build operations.
|
||||||
|
|
||||||
|
5. `openim::release::package_tarballs`: This function is used to package tarball files required for the release. These tarballs are usually used for distribution packages during the release.
|
||||||
|
|
||||||
|
6. `openim::release::upload_tarballs`: This function is used to upload the packaged tarball files, typically to a distribution platform or repository.
|
||||||
|
|
||||||
|
7. `git push origin ${VERSION}`: This line of command pushes the version tag to the remote Git repository's `origin` branch, marking this release in the version control system.
|
||||||
|
|
||||||
|
In the comments, you can see that there are some operations that are commented out, such as `openim::build::build_image`, `openim::release::github_release`, and `openim::release::generate_changelog`. These operations are related to building images, releasing to GitHub, and generating changelogs, and they can be enabled in the release process as needed.
|
||||||
|
|
||||||
|
Let's take a closer look at the function responsible for packaging the tarball files:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
function openim::release::package_tarballs() {
|
||||||
|
# Clean out any old releases
|
||||||
|
rm -rf "${RELEASE_STAGE}" "${RELEASE_TARS}" "${RELEASE_IMAGES}"
|
||||||
|
mkdir -p "${RELEASE_TARS}"
|
||||||
|
openim::release::package_src_tarball &
|
||||||
|
openim::release::package_client_tarballs &
|
||||||
|
openim::release::package_openim_manifests_tarball &
|
||||||
|
openim::release::package_server_tarballs &
|
||||||
|
openim::util::wait-for-jobs || { openim::log::error "previous tarball phase failed"; return 1; }
|
||||||
|
|
||||||
|
openim::release::package_final_tarball & # _final depends on some of the previous phases
|
||||||
|
openim::util::wait-for-jobs || { openim::log::error "previous tarball phase failed"; return 1; }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The `openim::release::package_tarballs()` function is responsible for packaging the tarball files required for the release. Here is the specific logic of this function:
|
||||||
|
|
||||||
|
1. `rm -rf "${RELEASE_STAGE}" "${RELEASE_TARS}" "${RELEASE_IMAGES}"`: First, the function removes any old release directories and files to ensure a clean starting state.
|
||||||
|
|
||||||
|
2. `mkdir -p "${RELEASE_TARS}"`: Next, it creates a directory `${RELEASE_TARS}` to store the packaged tarball files. If the directory doesn't exist, it will be created.
|
||||||
|
|
||||||
|
3. `openim::release::package_final_tarball &`: This is an asynchronous operation that depends on some of the previous phases. It is likely used to package the final tarball file, which includes the contents of all previous asynchronous operations.
|
||||||
|
|
||||||
|
4. `openim::util::wait-for-jobs`: It waits for all asynchronous operations to complete. If any of the previous asynchronous operations fail, an error will be returned.
|
||||||
@ -0,0 +1,173 @@
|
|||||||
|
#!/usr/bin/env python
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
|
||||||
|
#**************************************************************************
|
||||||
|
# Copyright (C) 2011, Paul Lutus *
|
||||||
|
# *
|
||||||
|
# This program is free software; you can redistribute it and/or modify *
|
||||||
|
# it under the terms of the GNU General Public License as published by *
|
||||||
|
# the Free Software Foundation; either version 2 of the License, or *
|
||||||
|
# (at your option) any later version. *
|
||||||
|
# *
|
||||||
|
# This program is distributed in the hope that it will be useful, *
|
||||||
|
# but WITHOUT ANY WARRANTY; without even the implied warranty of *
|
||||||
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
|
||||||
|
# GNU General Public License for more details. *
|
||||||
|
# *
|
||||||
|
# You should have received a copy of the GNU General Public License *
|
||||||
|
# along with this program; if not, write to the *
|
||||||
|
# Free Software Foundation, Inc., *
|
||||||
|
# 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. *
|
||||||
|
#**************************************************************************
|
||||||
|
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
|
PVERSION = '1.0'
|
||||||
|
|
||||||
|
|
||||||
|
class BeautifyBash:
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.tab_str = ' '
|
||||||
|
self.tab_size = 2
|
||||||
|
|
||||||
|
def read_file(self, fp):
|
||||||
|
with open(fp) as f:
|
||||||
|
return f.read()
|
||||||
|
|
||||||
|
def write_file(self, fp, data):
|
||||||
|
with open(fp, 'w') as f:
|
||||||
|
f.write(data)
|
||||||
|
|
||||||
|
def beautify_string(self, data, path=''):
|
||||||
|
tab = 0
|
||||||
|
case_stack = []
|
||||||
|
in_here_doc = False
|
||||||
|
defer_ext_quote = False
|
||||||
|
in_ext_quote = False
|
||||||
|
ext_quote_string = ''
|
||||||
|
here_string = ''
|
||||||
|
output = []
|
||||||
|
line = 1
|
||||||
|
for record in re.split('\n', data):
|
||||||
|
record = record.rstrip()
|
||||||
|
stripped_record = record.strip()
|
||||||
|
|
||||||
|
# collapse multiple quotes between ' ... '
|
||||||
|
test_record = re.sub(r'\'.*?\'', '', stripped_record)
|
||||||
|
# collapse multiple quotes between " ... "
|
||||||
|
test_record = re.sub(r'".*?"', '', test_record)
|
||||||
|
# collapse multiple quotes between ` ... `
|
||||||
|
test_record = re.sub(r'`.*?`', '', test_record)
|
||||||
|
# collapse multiple quotes between \` ... ' (weird case)
|
||||||
|
test_record = re.sub(r'\\`.*?\'', '', test_record)
|
||||||
|
# strip out any escaped single characters
|
||||||
|
test_record = re.sub(r'\\.', '', test_record)
|
||||||
|
# remove '#' comments
|
||||||
|
test_record = re.sub(r'(\A|\s)(#.*)', '', test_record, 1)
|
||||||
|
if(not in_here_doc):
|
||||||
|
if(re.search('<<-?', test_record)):
|
||||||
|
here_string = re.sub(
|
||||||
|
'.*<<-?\s*[\'|"]?([_|\w]+)[\'|"]?.*', '\\1', stripped_record, 1)
|
||||||
|
in_here_doc = (len(here_string) > 0)
|
||||||
|
if(in_here_doc): # pass on with no changes
|
||||||
|
output.append(record)
|
||||||
|
# now test for here-doc termination string
|
||||||
|
if(re.search(here_string, test_record) and not re.search('<<', test_record)):
|
||||||
|
in_here_doc = False
|
||||||
|
else: # not in here doc
|
||||||
|
if(in_ext_quote):
|
||||||
|
if(re.search(ext_quote_string, test_record)):
|
||||||
|
# provide line after quotes
|
||||||
|
test_record = re.sub(
|
||||||
|
'.*%s(.*)' % ext_quote_string, '\\1', test_record, 1)
|
||||||
|
in_ext_quote = False
|
||||||
|
else: # not in ext quote
|
||||||
|
if(re.search(r'(\A|\s)(\'|")', test_record)):
|
||||||
|
# apply only after this line has been processed
|
||||||
|
defer_ext_quote = True
|
||||||
|
ext_quote_string = re.sub(
|
||||||
|
'.*([\'"]).*', '\\1', test_record, 1)
|
||||||
|
# provide line before quote
|
||||||
|
test_record = re.sub(
|
||||||
|
'(.*)%s.*' % ext_quote_string, '\\1', test_record, 1)
|
||||||
|
if(in_ext_quote):
|
||||||
|
# pass on unchanged
|
||||||
|
output.append(record)
|
||||||
|
else: # not in ext quote
|
||||||
|
inc = len(re.findall(
|
||||||
|
'(\s|\A|;)(case|then|do)(;|\Z|\s)', test_record))
|
||||||
|
inc += len(re.findall('(\{|\(|\[)', test_record))
|
||||||
|
outc = len(re.findall(
|
||||||
|
'(\s|\A|;)(esac|fi|done|elif)(;|\)|\||\Z|\s)', test_record))
|
||||||
|
outc += len(re.findall('(\}|\)|\])', test_record))
|
||||||
|
if(re.search(r'\besac\b', test_record)):
|
||||||
|
if(len(case_stack) == 0):
|
||||||
|
sys.stderr.write(
|
||||||
|
'File %s: error: "esac" before "case" in line %d.\n' % (
|
||||||
|
path, line)
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
outc += case_stack.pop()
|
||||||
|
# sepcial handling for bad syntax within case ... esac
|
||||||
|
if(len(case_stack) > 0):
|
||||||
|
if(re.search('\A[^(]*\)', test_record)):
|
||||||
|
# avoid overcount
|
||||||
|
outc -= 2
|
||||||
|
case_stack[-1] += 1
|
||||||
|
if(re.search(';;', test_record)):
|
||||||
|
outc += 1
|
||||||
|
case_stack[-1] -= 1
|
||||||
|
# an ad-hoc solution for the "else" keyword
|
||||||
|
else_case = (
|
||||||
|
0, -1)[re.search('^(else)', test_record) != None]
|
||||||
|
net = inc - outc
|
||||||
|
tab += min(net, 0)
|
||||||
|
extab = tab + else_case
|
||||||
|
extab = max(0, extab)
|
||||||
|
output.append(
|
||||||
|
(self.tab_str * self.tab_size * extab) + stripped_record)
|
||||||
|
tab += max(net, 0)
|
||||||
|
if(defer_ext_quote):
|
||||||
|
in_ext_quote = True
|
||||||
|
defer_ext_quote = False
|
||||||
|
if(re.search(r'\bcase\b', test_record)):
|
||||||
|
case_stack.append(0)
|
||||||
|
line += 1
|
||||||
|
error = (tab != 0)
|
||||||
|
if(error):
|
||||||
|
sys.stderr.write(
|
||||||
|
'File %s: error: indent/outdent mismatch: %d.\n' % (path, tab))
|
||||||
|
return '\n'.join(output), error
|
||||||
|
|
||||||
|
def beautify_file(self, path):
|
||||||
|
error = False
|
||||||
|
if(path == '-'):
|
||||||
|
data = sys.stdin.read()
|
||||||
|
result, error = self.beautify_string(data, '(stdin)')
|
||||||
|
sys.stdout.write(result)
|
||||||
|
else: # named file
|
||||||
|
data = self.read_file(path)
|
||||||
|
result, error = self.beautify_string(data, path)
|
||||||
|
if(data != result):
|
||||||
|
# make a backup copy
|
||||||
|
self.write_file(path + '~', data)
|
||||||
|
self.write_file(path, result)
|
||||||
|
return error
|
||||||
|
|
||||||
|
def main(self):
|
||||||
|
error = False
|
||||||
|
sys.argv.pop(0)
|
||||||
|
if(len(sys.argv) < 1):
|
||||||
|
sys.stderr.write(
|
||||||
|
'usage: shell script filenames or \"-\" for stdin.\n')
|
||||||
|
else:
|
||||||
|
for path in sys.argv:
|
||||||
|
error |= self.beautify_file(path)
|
||||||
|
sys.exit((0, 1)[error])
|
||||||
|
|
||||||
|
# if not called as a module
|
||||||
|
if(__name__ == '__main__'):
|
||||||
|
BeautifyBash().main()
|
||||||
|
|
||||||
@ -0,0 +1,102 @@
|
|||||||
|
# Development of a Go-Based Conformity Checker for Project File and Directory Naming Standards
|
||||||
|
|
||||||
|
### 1. Project Overview
|
||||||
|
|
||||||
|
#### Project Name
|
||||||
|
|
||||||
|
- `GoConformityChecker`
|
||||||
|
|
||||||
|
#### Functionality Description
|
||||||
|
|
||||||
|
- Checks if the file and subdirectory names in a specified directory adhere to specific naming conventions.
|
||||||
|
- Supports specific file types (e.g., `.go`, `.yml`, `.yaml`, `.md`, `.sh`, etc.).
|
||||||
|
- Allows users to specify directories to be checked and directories to be ignored.
|
||||||
|
- More read https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/code-conventions.md
|
||||||
|
|
||||||
|
#### Naming Conventions
|
||||||
|
|
||||||
|
- Go files: Only underscores are allowed.
|
||||||
|
- YAML, YML, and Markdown files: Only hyphens are allowed.
|
||||||
|
- Directories: Only underscores are allowed.
|
||||||
|
|
||||||
|
### 2. File Structure
|
||||||
|
|
||||||
|
- `main.go`: Entry point of the program, handles command-line arguments.
|
||||||
|
- `checker/checker.go`: Contains the core logic.
|
||||||
|
- `config/config.go`: Parses and stores configuration information.
|
||||||
|
|
||||||
|
### 3. Core Code Design
|
||||||
|
|
||||||
|
#### main.go
|
||||||
|
|
||||||
|
- Parses command-line arguments, including the directory to be checked and directories to be ignored.
|
||||||
|
- Calls the `checker` module for checking.
|
||||||
|
|
||||||
|
#### config.go
|
||||||
|
|
||||||
|
- Defines a configuration structure, such as directories to check and ignore.
|
||||||
|
|
||||||
|
#### checker.go
|
||||||
|
|
||||||
|
- Iterates through the specified directory.
|
||||||
|
- Applies different naming rules based on file types and directory names.
|
||||||
|
- Records files or directories that do not conform to the standards.
|
||||||
|
|
||||||
|
### 4. Pseudocode Example
|
||||||
|
|
||||||
|
#### main.go
|
||||||
|
|
||||||
|
```go
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"GoConformityChecker/checker"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
// Parse command-line arguments
|
||||||
|
var targetDir string
|
||||||
|
var ignoreDirs string
|
||||||
|
flag.StringVar(&targetDir, "target", ".", "Directory to check")
|
||||||
|
flag.StringVar(&ignoreDirs, "ignore", "", "Directories to ignore")
|
||||||
|
flag.Parse()
|
||||||
|
|
||||||
|
// Call the checker
|
||||||
|
err := checker.CheckDirectory(targetDir, ignoreDirs)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Println("Error:", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### checker.go
|
||||||
|
|
||||||
|
```go
|
||||||
|
package checker
|
||||||
|
|
||||||
|
import (
|
||||||
|
// Import necessary packages
|
||||||
|
)
|
||||||
|
|
||||||
|
func CheckDirectory(targetDir, ignoreDirs string) error {
|
||||||
|
// Iterate through the directory, applying rules to check file and directory names
|
||||||
|
// Return any found errors or non-conformities
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Implementation Details
|
||||||
|
|
||||||
|
- **File and Directory Traversal**: Use Go's `path/filepath` package to traverse directories and subdirectories.
|
||||||
|
- **Naming Rules Checking**: Apply different regex expressions for naming checks based on file extensions.
|
||||||
|
- **Error Handling and Reporting**: Record files or directories that do not conform and report to the user.
|
||||||
|
|
||||||
|
### 6. Future Development and Extensions
|
||||||
|
|
||||||
|
- Support more file types and naming rules.
|
||||||
|
- Provide more detailed error reports, such as showing line numbers and specific naming mistakes.
|
||||||
|
- Add a graphical or web interface for non-command-line users.
|
||||||
|
|
||||||
|
The above is an overview of the entire project's design. Following this design, specific coding implementation can begin. Note that the actual implementation may need adjustments based on real-world conditions.
|
||||||
@ -1,10 +0,0 @@
|
|||||||
# See the OWNERS docs at https://go.k8s.io/owners
|
|
||||||
|
|
||||||
reviewers:
|
|
||||||
- cubxxw
|
|
||||||
- kubbot
|
|
||||||
approvers:
|
|
||||||
- cubxxw
|
|
||||||
labels:
|
|
||||||
- sig/testing
|
|
||||||
- sig/contributor-experience
|
|
||||||
Loading…
Reference in new issue