syf 发布的文章

最近开发javaagent应用，参照jps命令引入了sun.jvmstat包下的一些工具写了些逻辑，Maven打包时确实有些tricky，很容易打包后执行时找不到sun.jvmstat包下的一些类，记录下解决过程

1. dependency 引入

引入jdk内部依赖方法：定位.jar文件位置，scope设置为system引入

<!-- for jdk internal lib -->
<dependency>
    <groupId>com.sun</groupId>
    <artifactId>tools</artifactId>
    <version>1.8.0</version>
    <scope>system</scope>
    <systemPath>${java.home}/../lib/tools.jar</systemPath>
</dependency>

2. Assembly Plugin 配置

正常maven-compiler-plugins是没问题的，但是如果要打单个包，使用Assembly Plugin或Shade Plugin直接打包会有问题，因为system scope的缘故tools.jar内的库还是打不进去，此时需要添加descriptor配置：

<assembly
        xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
    <id>jar-with-all-dependencies</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <useProjectArtifact>true</useProjectArtifact>
            <unpack>true</unpack>
            <scope>runtime</scope>
        </dependencySet>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <unpack>true</unpack>
            <scope>system</scope>
        </dependencySet>
    </dependencySets>
</assembly>

保存为assembly.xml，在pom.xml的assembly-plugin插件配置段添加如下配置引入discriptor：

<configuration>
    <appendAssemblyId>false</appendAssemblyId>
    <descriptors>
        <descriptor>${basedir}/assembly.xml</descriptor>
    </descriptors>
</configuration>

再执行assembly打包，可以看到打出来的.jar文件内已经包含sun.jvmstat包内的相关class，可以正常执行了

参考

  1. https://stackoverflow.com/questions/3080437/jdk-tools-jar-as-maven-dependency

一、从arthas-spring-boot-starter开始

官方提供的Arthas Spring Boot Starter，以maven依赖的方式启动了arthas，attach自身进程后常驻；

以starter包为切入点查看arthas是如何进行attach和执行命令等一系列操作的

1.1 starter的configuration拉起AttachArthasClassloader

starter中的ArthasConfiguration类，指定在配置开启时加载attach.ArthasAgent并加载其init方法

init方法内的主要逻辑为启动AttachArthasClassloader：

com.taobao.arthas.agent.attach.ArthasAgent#init

if (instrumentation == null) {
    instrumentation = ByteBuddyAgent.install();
}
 
// 检查 arthasHome
if (arthasHome == null || arthasHome.trim().isEmpty()) {
    // 解压出 arthasHome
    URL coreJarUrl = this.getClass().getClassLoader().getResource("arthas-bin.zip");
    if (coreJarUrl != null) {
        File tempArthasDir = createTempDir();
        ZipUtil.unpack(coreJarUrl.openStream(), tempArthasDir);
        arthasHome = tempArthasDir.getAbsolutePath();
    } else {
        throw new IllegalArgumentException("can not getResources arthas-bin.zip from classloader: "
                + this.getClass().getClassLoader());
    }
}
 
// find arthas-core.jar
File arthasCoreJarFile = new File(arthasHome, ARTHAS_CORE_JAR);
if (!arthasCoreJarFile.exists()) {
    throw new IllegalStateException("can not find arthas-core.jar under arthasHome: " + arthasHome);
}
AttachArthasClassloader arthasClassLoader = new AttachArthasClassloader(
        new URL[] { arthasCoreJarFile.toURI().toURL() });
 
/**
 * <pre>
 * ArthasBootstrap bootstrap = ArthasBootstrap.getInstance(inst);
 * </pre>
 */
Class<?> bootstrapClass = arthasClassLoader.loadClass(ARTHAS_BOOTSTRAP);
Object bootstrap = bootstrapClass.getMethod(GET_INSTANCE, Instrumentation.class, Map.class).invoke(null,
        instrumentation, configMap);
boolean isBind = (Boolean) bootstrapClass.getMethod(IS_BIND).invoke(bootstrap);
if (!isBind) {
    String errorMsg = "Arthas server port binding failed! Please check $HOME/logs/arthas/arthas.log for more details.";
    throw new RuntimeException(errorMsg);
}

解压arthas-bin.zip包后通过AttachArthasClassloader加载，可以看到对应属性中bootstrap类是com.taobao.arthas.core.server.ArthasBootstrap，方法是getInstance，跳转到core中的Bootstrap类：

对应getInstance方法调了构造方法ArthasBootstrap，到这里arthas-core模块才真正起起来：

https://github.com/alibaba/arthas/blob/arthas-all-3.5.5/core/src/main/java/com/taobao/arthas/core/server/ArthasBootstrap.java#L350

其中最重要的流程是bind方法，启动了arthas server，关键逻辑在ShellServer的初始化：

1.2 ArthasBootstrap启动ShellServer

- 阅读剩余部分 -

生产环境前段时间neo4j图数据库总是在凌晨定时任务删除过期节点时报内存占用过高的告警，研究下怎么配置参数：

neo4j内存结构：

neo4j内存组成

OS保留：OS内存、Lucene索引缓存
页面缓存：图数据缓存、索引缓存
jvm堆：图查询、图管理、事务状态（可选）
事务状态
neo4j memory management

推荐的内存参数：

分配了3.6G jvm堆内存，2G 页面缓存，剩余2.6G空闲内存和 native memory 给 Lucene 和 Netty

一个收获：很多中间件jvm内存并不是一定需要占大头，例如某些MQ和缓存会使用 native memory 来避免jvm频繁垃圾回收带来的性能问题；以及像neo4j的Lucene引擎使用系统分页缓存

三个建议：

索引文件太大时，多剩点内存给OS
并发事务和update语句越多，分配的jvm堆内存也要越大（别超过31G）
jvm堆内存设置初始值和最大值一致，因为jvm更改堆大小时会触发full GC影响性能

syf@syf-ubuntu ~/o/neo4j-community-3.5.8 [1]> bin/neo4j-admin memrec --memory=8g
WARNING: Max 1024 open files allowed, minimum of 40000 recommended. See the Neo4j manual.
# Memory settings recommendation from neo4j-admin memrec:
#
# Assuming the system is dedicated to running Neo4j and has 8g of memory,
# we recommend a heap size of around 3600m, and a page cache of around 2g,
# and that about 2600m is left for the operating system, and the native memory
# needed by Lucene and Netty.
#
# Tip: If the indexing storage use is high, e.g. there are many indexes or most
# data indexed, then it might advantageous to leave more memory for the
# operating system.
#
# Tip: The more concurrent transactions your workload has and the more updates
# they do, the more heap memory you will need. However, don't allocate more
# than 31g of heap, since this will disable pointer compression, also known as
# "compressed oops", in the JVM and make less effective use of the heap.
#
# Tip: Setting the initial and the max heap size to the same value means the
# JVM will never need to change the heap size. Changing the heap size otherwise
# involves a full GC, which is desirable to avoid.
#
# Based on the above, the following memory settings are recommended:
dbms.memory.heap.initial_size=3600m
dbms.memory.heap.max_size=3600m
dbms.memory.pagecache.size=2g

参考：

https://neo4j.com/docs/operations-manual/3.5/performance/memory-configuration/

https://neo4j.com/docs/operations-manual/3.5/tools/neo4j-admin-memrec/

kubectl有些操作比较冗长，又经常需要使用，所以写了几个别名，进测试环境pod终端方便些，鉴权自行解决

使用例：

# 进入终端
podl qa sox
# 展示pod列表
pods qa sox
# 进入pod的某个container，如istio
podn qa istio sox-58654c84d-lnk4f
# 查看stdout，这个基本没啥用，除非有的异步线程抛出异常没打到logback指定的文件里
podlog qa sox
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
eval /c/Users/enmonster/miniconda3/Scripts/conda.exe "shell.fish" "hook" $argv | source
# <<< conda initialize <<<


function podl --argument-names nmsp app
    kubectl -n $nmsp exec -it (kubectl -n $nmsp get pod -l app=$app -o name) -c $app -- bash
end

function pods --argument-names nmsp app
    kubectl -n $nmsp get pods | grep $app
end

function podn --argument-names nmsp container pod
    kubectl -n $nmsp exec -it $pod -c $container -- bash
end

function podlog --argument-names nmsp app
    kubectl -n $nmsp logs -c $app (kubectl -n $nmsp get pod -l app=$app -o name)
end
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. ...
  7. 14