Docker+OpenResty+Lua文件上传与下载

Docker+OpenResty+Lua文件上传与下载,相关笔记。

前言

我们公司的服务器主要在阿里云上,之前公司未采用K8S进行进行集群的管理与部署时,主要采用传统的方式,单独的应用部署到单独的服务器上。在采用容器化方案之后,发现之前的常规方式不适用了,需要另外想办法实现。

之前开发人员有时需要登录服务器上查看日志,查看jvm堆栈信息,例如jmap,jstack命令等,之前方式是:购买服务器时选择自定义镜像,此镜像已安装好我们所需要的工具和软件包,例如:开发人员想要看服务器上jstack堆栈信息,方法是在阿里云OSS上新建bucket,把此文件上传到oss上,然后发送钉钉消息之类的提供下载链接,然后开发下载此文件进行信息查看:

1
2
3
$ /opt/jdk/bin/jstack -l $DUMP_ID > ${CURRENTDATE}_${CURRENTTIME}.jstack  && \
$ tar czvf ${APP_NAME}"_jstack_"${CURRENTDATE}_${CURRENTTIME}.tar.gz ${CURRENTDATE}_${CURRENTTIME}.jstack
$ ossutil cp -r ${APP_NAME}"_jstack_"${CURRENTDATE}_${CURRENTTIME}.tar.gz oss://bucket

目前我们已采用容器环境,遇到一个很坑的点,就是容器内不能使用ossutil工具!,oss工具暂时不支持容器内使用
所以采用了另外一种方案,用Nginx作为文件存储站,提供上传与下载

1
2
3
$ /opt/jdk/bin/jstack -l $DUMP_ID > ${CURRENTDATE}_${CURRENTTIME}.jstack  && \
$ tar czvf ${APP_NAME}"_jstack_"${CURRENTDATE}_${CURRENTTIME}.tar.gz ${CURRENTDATE}_${CURRENTTIME}.jstack
$ curl -u username:password -F "action=uploadfile" -F "file=@${APP_NAME}_jstack_${CURRENTDATE}_${CURRENTTIME}.tar.gz" https://uploadjstack.yourdomin.com/uploadfile

实现思路:OpenResty内嵌lua,lua模块内有savefile功能。

OpenResty® 是一个基于 Nginx 与 Lua 的高性能 Web 平台,其内部集成了大量精良的 Lua 库、第三方模块以及大多数的依赖项。用于方便地搭建能够处理超高并发、扩展性极高的动态 Web 应用、Web 服务和动态网关。

OpenResty镜像Dockerfile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
FROM centos:7

ARG RESTY_IMAGE_BASE="centos"
ARG RESTY_LUAROCKS_VERSION="2.4.4"
ARG RESTY_RPM_FLAVOR=""
ARG RESTY_RPM_VERSION="1.13.6.2-1.el7.centos"
ARG RESTY_RPM_ARCH="x86_64"

LABEL resty_luarocks_version="${RESTY_LUAROCKS_VERSION}"
LABEL resty_rpm_flavor="${RESTY_RPM_FLAVOR}"
LABEL resty_rpm_version="${RESTY_RPM_VERSION}"
LABEL resty_rpm_arch="${RESTY_RPM_ARCH}"

RUN yum-config-manager --add-repo https://openresty.org/package/${RESTY_IMAGE_BASE}/openresty.repo \
&& yum install -y \
gettext \
make \
openresty${RESTY_RPM_FLAVOR}-${RESTY_RPM_VERSION}.${RESTY_RPM_ARCH} \
openresty-opm-${RESTY_RPM_VERSION} \
openresty-resty-${RESTY_RPM_VERSION} \
unzip \
&& cd /tmp \
&& curl -fSL https://github.com/luarocks/luarocks/archive/${RESTY_LUAROCKS_VERSION}.tar.gz -o luarocks-${RESTY_LUAROCKS_VERSION}.tar.gz \
&& tar xzf luarocks-${RESTY_LUAROCKS_VERSION}.tar.gz \
&& cd luarocks-${RESTY_LUAROCKS_VERSION} \
&& ./configure \
--prefix=/usr/local/openresty/luajit \
--with-lua=/usr/local/openresty/luajit \
--lua-suffix=jit-2.1.0-beta3 \
--with-lua-include=/usr/local/openresty/luajit/include/luajit-2.1 \
&& make build \
&& make install \
&& cd /tmp \
&& rm -rf luarocks-${RESTY_LUAROCKS_VERSION} luarocks-${RESTY_LUAROCKS_VERSION}.tar.gz \
&& yum remove -y make \
&& yum clean all \
&& ln -sf /dev/stdout /usr/local/openresty/nginx/logs/access.log \
&& ln -sf /dev/stderr /usr/local/openresty/nginx/logs/error.log \
&& mkdir -p /usr/local/openresty/nginx/conf/lua \
&& chmod 755 /usr/local/openresty/nginx/html \
&& yum install httpd-tools -y \
&& htpasswd -bc /usr/local/openresty/nginx/conf/passwd username password

ARG RESTY_J="1"

# Add additional binaries into PATH for convenience
ENV PATH=$PATH:/usr/local/openresty/luajit/bin:/usr/local/openresty/nginx/sbin:/usr/local/openresty/bin

ENV LUA_PATH="/usr/local/openresty/site/lualib/?.ljbc;/usr/local/openresty/site/lualib/?/init.ljbc;/usr/local/openresty/lualib/?.ljbc;/usr/local/openresty/lualib/?/init.ljbc;/usr/local/openresty/site/lualib/?.lua;/usr/local/openresty/site/lualib/?/init.lua;/usr/local/openresty/lualib/?.lua;/usr/local/openresty/lualib/?/init.lua;./?.lua;/usr/local/openresty/luajit/share/luajit-2.1.0-beta3/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/local/openresty/luajit/share/lua/5.1/?.lua;/usr/local/openresty/luajit/share/lua/5.1/?/init.lua"

ENV LUA_CPATH="/usr/local/openresty/site/lualib/?.so;/usr/local/openresty/lualib/?.so;./?.so;/usr/local/lib/lua/5.1/?.so;/usr/local/openresty/luajit/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so;/usr/local/openresty/luajit/lib/lua/5.1/?.so"

# Copy nginx configuration files
COPY savefile1.lua /usr/local/openresty/nginx/conf/lua/
COPY nginx.conf /usr/local/openresty/nginx/conf/nginx.conf


CMD ["/usr/bin/openresty", "-g", "daemon off;"]

STOPSIGNAL SIGQUIT

nginx.conf配置文件:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

#user nobody;
user root;
worker_processes 1;

#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;

#pid logs/nginx.pid;


events {
worker_connections 1024;
}


http {
include mime.types;
default_type application/octet-stream;

#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';

#access_log logs/access.log main;
client_max_body_size 500M;
sendfile on;
#tcp_nopush on;

#keepalive_timeout 0;
keepalive_timeout 65;

#gzip on;

server {
listen 80;
server_name localhost;

#charset koi8-r;

#access_log logs/host.access.log main;

location / {
root html;
autoindex on;
autoindex_exact_size on;
autoindex_localtime on;
index index.html index.htm;
# auth_basic "Please input password";
# auth_basic_user_file /usr/local/openresty/nginx/conf/passwd;

}

location /uploadfile
{
auth_basic "Please input password";
auth_basic_user_file /usr/local/openresty/nginx/conf/passwd;

content_by_lua_file '/usr/local/openresty/nginx/conf/lua/savefile1.lua';
}

#error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}


}

savefile1.lua代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
package.path = '/usr/local/openresty/lualib/resty/?.lua;'
local upload = require "upload"

local chunk_size = 4096
local form = upload:new(chunk_size)
local file
local filelen=0
form:set_timeout(0) -- 1 sec
local filename

function get_filename(res)
local filename = ngx.re.match(res,'(.+)filename="(.+)"(.*)')
if filename then
return filename[2]
end
end

local osfilepath = "/usr/local/openresty/nginx/html/"
local i=0
while true do
local typ, res, err = form:read()
if not typ then
ngx.say("failed to read: ", err)
return
end
if typ == "header" then
if res[1] ~= "Content-Type" then
filename = get_filename(res[2])
if filename then
i=i+1
filepath = osfilepath .. filename
file = io.open(filepath,"w+")
if not file then
ngx.say("failed to open file ")
return
end
else
end
end
elseif typ == "body" then
if file then
filelen= filelen + tonumber(string.len(res))
file:write(res)
else
end
elseif typ == "part_end" then
if file then
file:close()
file = nil
ngx.say("file upload success")
end
elseif typ == "eof" then
break
else
end
end
if i==0 then
ngx.say("please upload at least one file!")
return
end

根据上述Dockerfile和相关文件构建docker镜像,加入到集群中去,配置相关域名转发即可。

调用脚本 gdDumpJstack

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#! /bin/bash
#gdDumpJstack
CURRENTDATE=`date +'%Y%m%d'`;
CURRENTTIME=`date +'%H%M%S'`;
DUMP_ID=`ps |grep -v grep |grep java|awk '{print $1}'`;
if [ -z "$DUMP_ID" ]; then
echo "cant find java process";
exit 1;
else
cd /tmp
/opt/jdk/bin/jstack -l $DUMP_ID > ${CURRENTDATE}_${CURRENTTIME}.jstack && \
tar czvf ${APP_NAME}"_jstack_"${CURRENTDATE}_${CURRENTTIME}.tar.gz ${CURRENTDATE}_${CURRENTTIME}.jstack
curl -u username:password -F "action=uploadfile" -F "file=@${APP_NAME}_jstack_${CURRENTDATE}_${CURRENTTIME}.tar.gz" https://uploadjstack.youdomain.com/uploadfile

sleep 1
echo "jstack download URL: https://uploadjstack.youdomain.com/${APP_NAME}_jstack_${CURRENTDATE}_${CURRENTTIME}.tar.gz";

rm -f *${CURRENTDATE}_${CURRENTTIME}.*
curl 'https://oapi.dingtalk.com/robot/send?access_token=tokenid' \
-H 'Content-Type: application/json' \
-d "{\"msgtype\": \"text\",
\"text\": {
\"content\": \"应用名:${APP_NAME},环境:${PROFILE} \n gdDumpJstack download URL:https://uploadjstack.youdomain.com/${APP_NAME}_jstack_${CURRENTDATE}_${CURRENTTIME}.tar.gz \"
}
}"
echo "."

fi

把此脚本加入到应用docker镜像,开发人员调用脚本即可通过钉钉消息发送的下载链接下载日志文件。

-------------本文结束感谢您的阅读-------------