BlueXIII's Blog

ApiMock相关选型

Posted on 2022-11-02

选型

spotlight 关注
apifox 关注
yapi
swagger-faker
easydoc
apizza
torna

torna安装

docker pull tanghc2020/torna:1.15.7

docker run --name torna --restart=always \
  -p 7700:7700 \
  -e JAVA_OPTS="-Xms256m -Xmx256m" \
  -e MYSQL_HOST="10.193.2.12:3306" \
  -e MYSQL_SCHEMA="torna" \
  -e MYSQL_USERNAME="root" \
  -e MYSQL_PASSWORD="yourpass" \
  -d tanghc2020/torna:1.15.7

http://10.193.2.12:7700

SSH双向隧道

Posted on 2022-11-02

参考文档

https://www.zsythink.net/archives/2450

脚本

# 1、通过本地端口建立ssh隧道转发请求
ssh -CNg -D8000 -p9922 root@10.80.8.238

# 2、ssh隧道代理本地端口
ssh -N -R 20001:127.0.0.1:20001 root@10.193.2.5

Sysbench测试

Posted on 2022-11-02

参考文档

安装

# 安装
# curl -s https://packagecloud.io/install/repositories/akopytov/sysbench/script.rpm.sh | sudo bash
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
sudo yum -y install sysbench

# LUA脚本目录
/usr/share/sysbench

MySQL压测

脚本

#  准备数据
sysbench \
--mysql-host=10.193.10.252 \
--mysql-port=3306 \
--mysql-db=sysbench \
--mysql-user=dev \
--mysql-password=yourpass \
/usr/share/sysbench/oltp_common.lua \
--tables=10 \
--table_size=100000 \
prepare

#  清理数据
sysbench \
--mysql-host=10.193.10.252 \
--mysql-port=3306 \
--mysql-db=sysbench \
--mysql-user=dev \
--mysql-password=yourpass \
/usr/share/sysbench/oltp_common.lua \
--tables=10 \
cleanup

#  测试
sysbench \
--mysql-host=10.193.10.252 \
--mysql-port=3306 \
--mysql-db=sysbench \
--mysql-user=dev \
--mysql-password=yourpass \
--threads=20 \
--time=60 \
--report-interval=5 \
/usr/share/sysbench/oltp_read_write.lua \
--tables=10 \
--table_size=100000 \
run

测试结果 10.193.10.252 写实例

[ 5s ] thds: 4 tps: 426.24 qps: 8533.27 (r/w/o: 5975.01/1704.97/853.29) lat (ms,95%): 10.09 err/s: 0.00 reconn/s: 0.00
[ 10s ] thds: 4 tps: 458.81 qps: 9181.27 (r/w/o: 6426.59/1837.05/917.63) lat (ms,95%): 9.56 err/s: 0.00 reconn/s: 0.00
[ 15s ] thds: 4 tps: 510.40 qps: 10205.85 (r/w/o: 7144.44/2040.61/1020.81) lat (ms,95%): 8.90 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 4 tps: 480.19 qps: 9604.44 (r/w/o: 6722.49/1921.57/960.38) lat (ms,95%): 8.90 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            131362
        write:                           37532
        other:                           18766
        total:                           187660
    transactions:                        9383   (468.95 per sec.)
    queries:                             187660 (9378.96 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          20.0069s
    total number of events:              9383

Latency (ms):
         min:                                    5.30
         avg:                                    8.53
         max:                                  120.79
         95th percentile:                        9.22
         sum:                                79990.85

Threads fairness:
    events (avg/stddev):           2345.7500/10.08
    execution time (avg/stddev):   19.9977/0.00

测试结果 10.193.10.250 Proxy

[ 5s ] thds: 4 tps: 270.87 qps: 5427.64 (r/w/o: 3801.41/1083.69/542.54) lat (ms,95%): 15.83 err/s: 0.00 reconn/s: 0.00
[ 10s ] thds: 4 tps: 257.42 qps: 5146.09 (r/w/o: 3600.95/1030.30/514.85) lat (ms,95%): 16.12 err/s: 0.00 reconn/s: 0.00
[ 15s ] thds: 4 tps: 261.62 qps: 5233.32 (r/w/o: 3664.43/1045.66/523.23) lat (ms,95%): 16.41 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 4 tps: 271.79 qps: 5435.24 (r/w/o: 3804.49/1087.17/543.58) lat (ms,95%): 15.55 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            74382
        write:                           21252
        other:                           10626
        total:                           106260
    transactions:                        5313   (265.48 per sec.)
    queries:                             106260 (5309.53 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          20.0115s
    total number of events:              5313

Latency (ms):
         min:                                    9.53
         avg:                                   15.06
         max:                                  120.29
         95th percentile:                       16.12
         sum:                                80021.11

Threads fairness:
    events (avg/stddev):           1328.2500/4.15
    execution time (avg/stddev):   20.0053/0.00

TiDB

mysql://10.193.0.8:4000 root/yourpass

脚本

# 准备
sysbench \
--mysql-host=10.193.0.8 \
--mysql-port=4000 \
--mysql-db=sysbench \
--mysql-user=root \
--mysql-password=yourpass \
/usr/share/sysbench/oltp_common.lua \
--tables=10 \
--table_size=100000 \
prepare

#  测试
sysbench \
--mysql-host=10.193.0.8 \
--mysql-port=4000 \
--mysql-db=sysbench \
--mysql-user=root \
--mysql-password=yourpass \
--threads=20 \
--time=60 \
--report-interval=5 \
/usr/share/sysbench/oltp_read_write.lua \
--tables=10 \
--table_size=100000 \
run

测试结果

[ 5s ] thds: 20 tps: 469.21 qps: 9454.95 (r/w/o: 6623.50/1889.03/942.42) lat (ms,95%): 130.13 err/s: 0.00 reconn/s: 0.00
[ 10s ] thds: 20 tps: 494.81 qps: 9882.80 (r/w/o: 6920.34/1972.84/989.62) lat (ms,95%): 106.75 err/s: 0.00 reconn/s: 0.00
[ 15s ] thds: 20 tps: 441.62 qps: 8837.18 (r/w/o: 6185.26/1768.68/883.24) lat (ms,95%): 132.49 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 20 tps: 503.80 qps: 10065.62 (r/w/o: 7044.81/2013.60/1007.20) lat (ms,95%): 94.10 err/s: 0.00 reconn/s: 0.00
[ 25s ] thds: 20 tps: 439.01 qps: 8783.70 (r/w/o: 6149.27/1756.02/878.41) lat (ms,95%): 130.13 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 20 tps: 383.00 qps: 7660.47 (r/w/o: 5362.45/1532.01/766.01) lat (ms,95%): 137.35 err/s: 0.00 reconn/s: 0.00
[ 35s ] thds: 20 tps: 439.79 qps: 8793.80 (r/w/o: 6156.66/1757.56/879.58) lat (ms,95%): 130.13 err/s: 0.00 reconn/s: 0.00
[ 40s ] thds: 20 tps: 467.00 qps: 9337.01 (r/w/o: 6534.81/1868.60/933.60) lat (ms,95%): 118.92 err/s: 0.00 reconn/s: 0.00
[ 45s ] thds: 20 tps: 418.60 qps: 8381.86 (r/w/o: 5868.84/1675.41/837.61) lat (ms,95%): 134.90 err/s: 0.00 reconn/s: 0.00
[ 50s ] thds: 20 tps: 459.79 qps: 9196.03 (r/w/o: 6435.68/1840.77/919.58) lat (ms,95%): 101.13 err/s: 0.00 reconn/s: 0.00
[ 55s ] thds: 20 tps: 463.01 qps: 9255.66 (r/w/o: 6479.38/1850.25/926.03) lat (ms,95%): 108.68 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 20 tps: 415.40 qps: 8288.56 (r/w/o: 5799.77/1658.79/830.00) lat (ms,95%): 137.35 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            377944
        write:                           107984
        other:                           53992
        total:                           539920
    transactions:                        26996  (449.31 per sec.)
    queries:                             539920 (8986.21 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          60.0817s
    total number of events:              26996

Latency (ms):
         min:                                   20.72
         avg:                                   44.47
         max:                                  368.03
         95th percentile:                      130.13
         sum:                              1200584.83

Threads fairness:
    events (avg/stddev):           1349.8000/15.89
    execution time (avg/stddev):   60.0292/0.02

CPU

1	sysbench cpu --threads=40 --events=10000 --cpu-max-prime=20000 run

磁盘

脚本

sysbench fileio help

# 准备
sysbench fileio --file-num=5 --file-total-size=1G prepare

# 测试
sysbench fileio \
--events=5000 \
--threads=16 \
--file-num=5 \
--file-total-size=1G \
--file-test-mode=rndrw \
--file-fsync-freq=0 \
--file-block-size=16384 \
run

结果

File operations:
    reads/s:                      33380.09
    writes/s:                     22271.94
    fsyncs/s:                     890.43

Throughput:
    read, MiB/s:                  521.56
    written, MiB/s:               348.00

General statistics:
    total time:                          0.0881s
    total number of events:              5000

Latency (ms):
         min:                                    0.00
         avg:                                    0.02
         max:                                    4.30
         95th percentile:                        0.04
         sum:                                  116.73

Threads fairness:
    events (avg/stddev):           312.5000/132.74
    execution time (avg/stddev):   0.0073/0.00

Docker

https://hub.docker.com/r/severalnines/sysbench/

脚本

# 拉镜像
docker pull severalnines/sysbench

# 磁盘准备
docker run -it \
--name=sysbench \
severalnines/sysbench \
bash

结果

File operations:
    reads/s:                      6408.42
    writes/s:                     4272.28
    fsyncs/s:                     170.89

Throughput:
    read, MiB/s:                  100.13
    written, MiB/s:               66.75

General statistics:
    total time:                          0.4660s
    total number of events:              5000

Latency (ms):
         min:                                    0.00
         avg:                                    0.02
         max:                                    3.48
         95th percentile:                        0.04
         sum:                                   95.95

Threads fairness:
    events (avg/stddev):           312.5000/219.91
    execution time (avg/stddev):   0.0060/0.00

SwarmPit部署笔记

Posted on 2022-11-02

官网

https://swarmpit.io/

参考文档

https://www.bboy.app/2020/10/10/%E4%BD%BF%E7%94%A8swarmpit%E6%9D%A5%E7%9B%91%E6%8E%A7%E4%BD%A0%E7%9A%84swarm%E9%9B%86%E7%BE%A4/

安装

# 注意添加--network host
docker run -it --rm \
  --name swarmpit-installer \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --network host \
swarmpit/install:1.9

界面

http://10.193.10.201:888
admin/yourpass

rsyslog远程日志

Posted on 2022-11-02

参考文档

http://docsv3.yiqiyun.net.cn/network/loadbalancer/manual/lb_user_guide/#rsyslog-%E8%BF%9C%E7%AB%AF%E4%BA%91%E6%9C%8D%E5%8A%A1%E5%99%A8

服务器

10.193.10.54

部署

vi /etc/rsyslog.conf
$ModLoad imudp
$UDPServerAddress 0.0.0.0
$UDPServerRun 514

vi /etc/rsyslog.d/haproxy.conf
local0.* /data/rsyslog/haproxy.log

systemctl restart rsyslog
lsof -i:514

查看日志

1	tail -f /data/rsyslog/haproxy.log

DockerSwarm子节点配置

Posted on 2022-11-02

Docker相关配置

vi /etc/hosts

1 2	10.193.10.15 artifactory.sdecloud.com 10.133.0.131 www.zykjfw.cn

vi /etc/docker/daemon.json

{
  "registry-mirrors":["https://r9xxm8z8.mirror.aliyuncs.com"],
  "insecure-registries":["artiServer:8082","10.193.10.15:8082","artifactory.sdecloud.com:8443","artifactory.sdecloud.com:8082","10.193.10.15:8443"]
}


# 重启
systemctl daemon-reload
systemctl restart docker

# 登录
docker login artifactory.sdecloud.com:8443
admin/Lzqzhu1241jie

# 测试
docker pull artifactory.sdecloud.com:8443/docker-local/h5:RELEASE-1.491

新增节点

# master上查看token
docker swarm join-token worker

# 新节点加入master
docker swarm join --token SWMTKN-1-18pycq92hlpw4q9okdh074d8y5etztku84312apwukehvqmgb6-76156tq10bc8kbn5oq38vn6lr 10.193.10.20:2377

# master上查看状态
docker node ls

DockerStack部署

Posted on 2022-11-02

服务器

10.193.10.20
compose脚本路径: /root/zhongyuan/stacks

注意事项

H5应用比较特殊，需要在每个节点的/root/zhongyuan/volumes目录入，放置node_modules，以供挂载
export的变量为镜像名称，最后面的版本号是jenkins的buildId，查询地址: https://artifactory.sdecloud.com:8443/ui/repos/tree/General/docker-local

Stack手工部署

1 2	export SERVICES_pay_SERVICE_IMAGE=artifactory.sdecloud.com:8443/docker-local/pay-service-impl:RELEASE-1.15 docker stack deploy -c pay-service.yml pay-service

验证

1 2	docker service ls docker service ps bridge-service_bridge-service

URL测试

http://10.193.10.20:8021
http://10.193.10.20:18082

V2Ray端口复用

Posted on 2022-11-02

官网及下载

跳板机

使用10.80.8.237
复用Nginx的8080端口
方式为vmess-over-websocket

V2ray服务器搭建

v2ray安装路径为/opt/v2ray
nginx配置文件路径为/dubhe/nginx/conf

配置服务端config.json，vi config.json:

{
  "log": {
    "access": "/opt/v2ray/access.log",
    "error": "/opt/v2ray/error.log",
    "loglevel": "debug"
  },
  "inbounds": [{
    "port": 31313,
    "listen": "172.17.0.1",
    "protocol": "vmess",
    "settings": {
      "clients": [{
        "id": "7255a312-be4b-9315-65c9-123412341234",
        "alterId": 64
      }]
    },
    "streamSettings": {
      "network": "ws",
      "wsSettings": {
        "path": "/ray/"
      }
    }
  }],
  "outbounds": [{
    "protocol": "freedom",
    "settings": {}
  }]
}

启动v2ray:

1	./v2ray --config config.json

配置Nginx的default.conf:

location /ray {
    proxy_redirect off;
    proxy_pass http://172.17.0.1:31313;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

重启Nginx:

1	docker restart nginx

本机DockerEasyConnect启动

docker run --name ecweifang --device /dev/net/tun --cap-add NET_ADMIN -itd -p 127.0.0.1:7011:1080 -e EC_VER=7.6.3 -e CLI_OPTS="-d https://xx.xx.xx.xx -u username -p password" hagb/docker-easyconnect:cli

占用端口7011

本机Proxifier配置

v2ray进程 + 10.80.8.237网段 –> 127.0.0.1:7011
任意进程 + 10.80.8.*网段 –> 127.0.0.1:7011

V2ray客户端配置

vi weifang.json

{
  "inbounds" : [
    {
      "listen" : "127.0.0.1",
      "port" : 7012,
      "protocol" : "socks",
      "tag" : "socksinbound",
      "settings" : {
        "auth" : "noauth",
        "udp" : false,
        "ip" : "127.0.0.1"
      }
    }
  ],
  "outbounds" : [
    {
      "protocol" : "vmess",
      "settings" : {
        "vnext" : [
          {
            "address" : "10.80.8.237",
            "users" : [
              {
                "id" : "7255a312-be4b-9315-65c9-123412341234",
                "alterId" : 64,
                "security" : "auto",
                "level" : 0
              }
            ],
            "port" : 8080
          }
        ]
      },
      "streamSettings" : {
        "wsSettings" : {
          "path" : "\/ray\/",
          "headers" : {

          }
        },
        "tlsSettings" : {
          "allowInsecure" : false,
          "alpn" : [
            "http\/1.1"
          ],
          "serverName" : "",
          "allowInsecureCiphers" : false
        },
        "security" : "none",
        "network" : "ws"
      }
    }
  ]
}

本机客户端启动:

1	./v2ray --config weifang.json

占用端口7012

Tomcat优化配置

Posted on 2022-11-02

server.xml测试

Connector->maxThreads，在64核下设置成200最优，设300或更高反而增加CPU切换时间大幅降低TPS。查资料默认值就是200，此参数不用设置。
Connector->minSpareThreads，设成200或直接用默认值25，没有区别
Connector->maxConnections,

server.xml

<?xml version="1.0" encoding="UTF-8"?>
<Server port="8005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">

<!-- <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="1024" minSpareThreads="128"/> -->

<!-- <Connector executor="tomcatThreadPool"
           protocol="org.apache.coyote.http11.Http11Nio2Protocol"
           port="8080"
           maxConnections="40000"
           acceptCount="40000"
           enableLookups="false"
           disableUploadTimeout="true"
           useSendfile="true"
           connectionTimeout="60000"
           redirectPort="8443"
           URIEncoding="UTF-8"
           compression="on"
           compressionMinSize="1024"
           compressibleMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript"
           /> -->
  <Connector port="8080" protocol="HTTP/1.1"
            connectionTimeout="20000" 
            maxConnections="20000"
            redirectPort="8443" />
    <Engine name="Catalina" defaultHost="localhost">
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

Redis集群搭建

Posted on 2022-11-02

参考文档

redis.conf配置

创建6个目录，将/etc/redis.conf拷贝至其中

# 端口号，每个目录都不同
port 7000
# 开启集群模式
cluster-enabled yes
#节点超时实际，单位毫秒
cluster-node-timeout 5000
#集群内部配置文件，绝对路径
cluster-config-file ~/opt/redis-cluster/7000/nodes.conf
# 启动 AOF
appendonly yes
# 密码
requirepass 123456
# 主从复制密码(只在3个从节点上配置)
masterauth 123456

简易操作步骤

# 启动6个redis节点
nohup redis-server ~/opt/redis-cluster/7000/redis.conf > 7000.log 2>&1 &
nohup redis-server ~/opt/redis-cluster/7001/redis.conf > 7001.log 2>&1 &
nohup redis-server ~/opt/redis-cluster/7002/redis.conf > 7002.log 2>&1 &
nohup redis-server ~/opt/redis-cluster/7003/redis.conf > 7003.log 2>&1 &
nohup redis-server ~/opt/redis-cluster/7004/redis.conf > 7004.log 2>&1 &
nohup redis-server ~/opt/redis-cluster/7005/redis.conf > 7005.log 2>&1 &

# 登录7000，关联其它节点
redis-cli -a 123456 -p 7000
> cluster meet 127.0.0.1 7001
> cluster meet 127.0.0.1 7002
> cluster meet 127.0.0.1 7003
> cluster meet 127.0.0.1 7004
> cluster meet 127.0.0.1 7005

# 查看集群状态，获取节点ID
redis-cli -a 123456 -p 7000 cluster nodes

# 配置主从复制
redis-cli -a 123456 -p 7003 cluster replicate 21509e5b757d8bae0678efcdb525f59def83674e  # 7000的ID
redis-cli -a 123456 -p 7004 cluster replicate ebb502cda79909b50350be31c3ffe96e9177bafd  # 7001的ID
redis-cli -a 123456 -p 7005 cluster replicate 2aa5a4c37c92f86fdae681dd8c361fee3a99a7af  # 7002的ID

# 检查并随机分配slots
redis-cli -a 123456 --cluster check 127.0.0.1:7000 
redis-cli -a 123456 --cluster fix 127.0.0.1:7000

配置文件示例

https://bluexiii-files.oss-cn-beijing.aliyuncs.com/redis.conf