ApiManager 的分析

Truman - Sep 7 - - Dev Community

WEB MVC的一些基础配置 中,ApiManager类是通过内部的InspectConfiguration类引用的。InspectConfiguration类使用了@Configuration和@ConditionalOnClass注解,表明当Spring上下文中存在RedisTemplate和InterceptorRegistry类时,会自动配置ApiManager。

1. 在InspectConfiguration中引用ApiManager

@Configuration
@ConditionalOnClass({RedisTemplate.class, InterceptorRegistry.class})
@EnableConfigurationProperties(ApiControlProperties.class)
static class InspectConfiguration {

    // 配置ApiManager Bean,负责管理API统计数据
    @Bean
    public ApiManager handlerMethodRedisRegistry(
            RedisTemplate<String, Object> redisTemplate,
            RequestMappingHandlerMapping requestMappingHandlerMapping) {
        return new ApiManager(redisTemplate, requestMappingHandlerMapping);
    }

    // 配置ApiControlInterceptor的自定义拦截器注册器,添加拦截器到注册器
    @Bean
    InterceptorRegistryCustomizer apiControlInterceptorCustomizer(ApiControlProperties apiControlProperties,
                                                                  @Autowired(required = false) CacheControl cacheControl,
                                                                  ApplicationContext applicationContext) {
        return registry -> registry.addInterceptor(new ApiControlInterceptor(apiControlProperties, cacheControl, applicationContext))
                .order(Ordered.HIGHEST_PRECEDENCE + 11) // 设置拦截器的优先级
                .addPathPatterns("/**"); // 设置拦截的路径模式
    }
}

Enter fullscreen mode Exit fullscreen mode

解析

1. @ConditionalOnClass:

  • 这个注解表示InspectConfiguration配置类仅在RedisTemplate和InterceptorRegistry这两个类存在于类路径时才会生效。这意味着只有在应用程序使用Redis并且包含Spring Web MVC时,才会创建ApiManager的Bean。

2. handlerMethodRedisRegistry 方法:

  • 该方法通过@Bean注解声明了一个ApiManager的Bean实例,它通过构造函数接收两个参数:
    • RedisTemplate:用于与Redis进行交互。
    • RequestMappingHandlerMapping:Spring MVC中的映射处理器,用于获取所有Handler方法信息。
  • 在handlerMethodRedisRegistry方法中,ApiManager被创建并注册到Spring容器中,作为一个可注入的Bean。这个ApiManager实例负责处理与API管理相关的逻辑,包括未使用API的管理、请求的统计和推送。

3. apiControlInterceptorCustomizer 方法:

  • 该方法配置了一个自定义的拦截器注册器,返回了一个InterceptorRegistryCustomizer接口的实现。
  • 在这个实现中,ApiControlInterceptor拦截器被添加到了Spring的拦截器链中,优先级非常高(Ordered.HIGHEST_PRECEDENCE + 11),并且该拦截器应用于所有请求路径(/**)。
  • ApiControlInterceptor会利用ApiManager来进行API请求的监控和统计。

ApiManager class

public class ApiManager {

    private final BoundSetOperations<String, Object> unusedApis;

    private final BoundHashOperations<String, String, Number> apiStatisticsHashOps;

    private final RedisTemplate<String, Object> redisTemplate;

    private volatile long lastSetExpireTime;

    public ApiManager(RedisTemplate<String, Object> redisTemplate, RequestMappingHandlerMapping rmhm) {

        this.redisTemplate = redisTemplate;

        String name = Objects.requireNonNull(SpringContextHolder.getRedisPrefixByApplicationName());
        this.unusedApis = redisTemplate.boundSetOps("entropy-reduction:unused-apis:" + name);
        this.apiStatisticsHashOps = redisTemplate.boundHashOps(name + ":api-statistics");

        HashOperations<String, String, String> hashOps = redisTemplate.opsForHash();
        String registerKey = "entropy-reduction:unused-apis:register";
        if (hashOps.get(registerKey, name) == null) {
            /**
             * 熵减工程之去除无用接口
             */
            hashOps.put(registerKey, name, DateFormatUtils.format(new Date(), "yyyy-MM-dd HH:mm:ss"));

            Object[] handlerMethods = rmhm.getHandlerMethods().values().stream().map(hm -> {
                Method method = hm.getMethod();
                return method.getDeclaringClass().getName() + "." + method.getName();
            }).distinct().filter(item -> item.startsWith("com.example")).toArray();

            unusedApis.add(handlerMethods);
        }
    }

    /**
     * 把请求存起来分析次数和权重
     */
    public void requestToRedis(String deviceId, String host, String clientType) {
        if (StringUtils.isBlank(deviceId)
                || StringUtils.isBlank(clientType)
                || StringUtils.isBlank(host)) {
            return;
        }

        if(DeviceUtils.isApp(clientType)) {
            String formattedDate = DateFormatUtils.format(new Date(), "yyyyMMdd");

            BoundHashOperations<String, String, Number> statisticalWeightHashOps =
                    redisTemplate.boundHashOps("project-common-core:ApiManager:requestToRedis:requestTotal:" + formattedDate);

            statisticalWeightHashOps.increment(clientType + " " + host, 1);

            if (System.currentTimeMillis() - lastSetExpireTime > 3600_000) {
                lastSetExpireTime = System.currentTimeMillis();
                statisticalWeightHashOps.expire(7, TimeUnit.DAYS);
            }

        }
    }

    public void markAsUsed(Set<String> calledApis) {
        unusedApis.remove(calledApis.toArray());
    }


    public Map<String, ApiStatistics> pushAndFetchApiStatistics(Map<String, ApiStatistics> input) {
        for (Map.Entry<String, ApiStatistics> entry : input.entrySet()) {
            this.apiStatisticsHashOps.increment(entry.getKey() + "-duration", entry.getValue().getTotalDuration());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-total", entry.getValue().getTotalCount());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-accept", entry.getValue().getTotalAcceptCount());
            this.apiStatisticsHashOps.increment(entry.getKey() + "-reject", entry.getValue().getTotalRejectCount());
        }

        Map<String, Number> entries = this.apiStatisticsHashOps.entries();

        if (entries != null) {
            Map<String, ApiStatistics> output = new HashMap<>();

            entries.forEach((key, value) -> {
                String[] segments = key.split("-");
                ApiStatistics apiCall = new ApiStatistics();
                switch (segments[1]) {
                    case "duration":
                        apiCall.setTotalDuration(value);
                        break;
                    case "total":
                        apiCall.setTotalCount(value);
                        break;
                    case "accept":
                        apiCall.setTotalAcceptCount(value);
                        break;
                    case "reject":
                        apiCall.setTotalRejectCount(value);
                        break;
                }

                ApiStatistics absent = output.putIfAbsent(segments[0], apiCall);
                if (absent != null) {
                    absent.incrementTotalCount(apiCall.getTotalCount());
                    absent.incrementDuration(apiCall.getTotalDuration());
                    absent.incrementReject(apiCall.getTotalRejectCount());
                    absent.incrementAccept(apiCall.getTotalAcceptCount());
                }
            });
            StringBuilder sb = new StringBuilder();
            output.forEach((key, value) -> {
                sb.append(key).append("\n");
                sb.append("\t-totalCount: ").append(value.getTotalCount()).append("\n");
                sb.append("\t-totalDuration: ").append(value.getTotalDuration()).append("ms\n");
                sb.append("\t-avgDuration: ").append(value.getAvgDuration()).append("ms\n");
                sb.append("\t-totalReject: ").append(value.getTotalRejectCount()).append("\n");
                sb.append("\t-totalAccept: ").append(value.getTotalAcceptCount()).append("\n\n");
                sb.append("-----------------------------------------------------------------\n");
            });
            try {
                FileUtils.writeStringToFile(new File("api-stat.txt"), sb.toString(), "UTF-8");
            } catch (Exception ignore) {
            }
            return output;
        } else {
            return new HashMap<>(input);
        }
    }

}
Enter fullscreen mode Exit fullscreen mode

1. 类成员变量

private final BoundSetOperations<String, Object> unusedApis;
private final BoundHashOperations<String, String, Number> apiStatisticsHashOps;
private final RedisTemplate<String, Object> redisTemplate;
private volatile long lastSetExpireTime;
Enter fullscreen mode Exit fullscreen mode
  • unusedApis: 使用BoundSetOperations接口操作Redis中的一个集合,用于存储未使用的API。
  • apiStatisticsHashOps: 使用BoundHashOperations接口操作Redis中的哈希表,用于存储每个API的统计信息。
  • redisTemplate: 用于与Redis交互的模板,提供操作Redis的各种方法。
  • lastSetExpireTime: 存储上次设置Redis键过期时间的时间戳,用于控制过期时间的设置频率。

2. 构造方法

public ApiManager(RedisTemplate<String, Object> redisTemplate, RequestMappingHandlerMapping rmhm) {
    this.redisTemplate = redisTemplate;

    String name = Objects.requireNonNull(SpringContextHolder.getRedisPrefixByApplicationName());
    this.unusedApis = redisTemplate.boundSetOps("entropy-reduction:unused-apis:" + name);
    this.apiStatisticsHashOps = redisTemplate.boundHashOps(name + ":api-statistics");

    HashOperations<String, String, String> hashOps = redisTemplate.opsForHash();
    String registerKey = "entropy-reduction:unused-apis:register";
    if (hashOps.get(registerKey, name) == null) {
        /**
         * 熵减工程之去除无用接口
         */
        hashOps.put(registerKey, name, DateFormatUtils.format(new Date(), "yyyy-MM-dd HH:mm:ss"));

        Object[] handlerMethods = rmhm.getHandlerMethods().values().stream().map(hm -> {
            Method method = hm.getMethod();
            return method.getDeclaringClass().getName() + "." + method.getName();
        }).distinct().filter(item -> item.startsWith("com.example")).toArray();

        unusedApis.add(handlerMethods);
    }
}
Enter fullscreen mode Exit fullscreen mode
  • redisTemplate初始化:构造函数接收RedisTemplate和RequestMappingHandlerMapping作为参数,初始化与Redis相关的操作对象unusedApis和apiStatisticsHashOps。
  • 初始化未使用的API集合:
    • 通过RequestMappingHandlerMapping获取所有注册的Handler方法,并将其作为候选未使用的API。
    • 如果Redis中不存在当前应用的未使用API的记录,则将其注册到Redis中,目的是进行API的熵减,即移除不再使用的API。

3. 请求信息存储方法

public void requestToRedis(String deviceId, String host, String clientType) {
    if (StringUtils.isBlank(deviceId) || StringUtils.isBlank(clientType) || StringUtils.isBlank(host)) {
        return;
    }

    if (DeviceUtils.isApp(clientType)) {
        String formattedDate = DateFormatUtils.format(new Date(), "yyyyMMdd");

        BoundHashOperations<String, String, Number> statisticalWeightHashOps =
                redisTemplate.boundHashOps("project-common-core:ApiManager:requestToRedis:requestTotal:" + formattedDate);

        statisticalWeightHashOps.increment(clientType + " " + host, 1);

        if (System.currentTimeMillis() - lastSetExpireTime > 3600_000) {
            lastSetExpireTime = System.currentTimeMillis();
            statisticalWeightHashOps.expire(7, TimeUnit.DAYS);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

功能解析:

  • 请求参数的存储:
    • 方法检查传入的deviceId、host和clientType是否为空,如果为空则直接返回。
    • 如果clientType符合应用客户端的要求,则将请求数据存储到Redis中的哈希结构中,以统计该客户端的请求总数。
  • 过期时间设置:
    • 每隔一小时会设置Redis键的过期时间为7天,以确保统计数据不会无限增长。

4. 标记使用的API

public void markAsUsed(Set<String> calledApis) {
    unusedApis.remove(calledApis.toArray());
}
Enter fullscreen mode Exit fullscreen mode
  • 移除已使用的API:
    • 方法接收一个已调用的API集合calledApis,并将其从unusedApis集合中移除。这一步是为了更新Redis中未使用API的记录,避免将活跃的API误认为未使用。

5. 推送与获取API统计信息

public Map<String, ApiStatistics> pushAndFetchApiStatistics(Map<String, ApiStatistics> input) {
    for (Map.Entry<String, ApiStatistics> entry : input.entrySet()) {
        this.apiStatisticsHashOps.increment(entry.getKey() + "-duration", entry.getValue().getTotalDuration());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-total", entry.getValue().getTotalCount());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-accept", entry.getValue().getTotalAcceptCount());
        this.apiStatisticsHashOps.increment(entry.getKey() + "-reject", entry.getValue().getTotalRejectCount());
    }

    Map<String, Number> entries = this.apiStatisticsHashOps.entries();

    if (entries != null) {
        Map<String, ApiStatistics> output = new HashMap<>();

        entries.forEach((key, value) -> {
            String[] segments = key.split("-");
            ApiStatistics apiCall = new ApiStatistics();
            switch (segments[1]) {
                case "duration":
                    apiCall.setTotalDuration(value);
                    break;
                case "total":
                    apiCall.setTotalCount(value);
                    break;
                case "accept":
                    apiCall.setTotalAcceptCount(value);
                    break;
                case "reject":
                    apiCall.setTotalRejectCount(value);
                    break;
            }

            ApiStatistics absent = output.putIfAbsent(segments[0], apiCall);
            if (absent != null) {
                absent.incrementTotalCount(apiCall.getTotalCount());
                absent.incrementDuration(apiCall.getTotalDuration());
                absent.incrementReject(apiCall.getTotalRejectCount());
                absent.incrementAccept(apiCall.getTotalAcceptCount());
            }
        });

        StringBuilder sb = new StringBuilder();
        output.forEach((key, value) -> {
            sb.append(key).append("\n");
            sb.append("\t-totalCount: ").append(value.getTotalCount()).append("\n");
            sb.append("\t-totalDuration: ").append(value.getTotalDuration()).append("ms\n");
            sb.append("\t-avgDuration: ").append(value.getAvgDuration()).append("ms\n");
            sb.append("\t-totalReject: ").append(value.getTotalRejectCount()).append("\n");
            sb.append("\t-totalAccept: ").append(value.getTotalAcceptCount()).append("\n\n");
            sb.append("-----------------------------------------------------------------\n");
        });
        try {
            FileUtils.writeStringToFile(new File("api-stat.txt"), sb.toString(), "UTF-8");
        } catch (Exception ignore) {
        }
        return output;
    } else {
        return new HashMap<>(input);
    }
}
Enter fullscreen mode Exit fullscreen mode
  • 推送统计数据到Redis:
    • 方法遍历输入的ApiStatistics对象,将每个API的统计数据(包括请求持续时间、总请求数、接受请求数、拒绝请求数)推送到Redis中的哈希结构中。
  • 从Redis获取最新的API统计数据:
    • 从Redis中获取统计数据后,构造一个新的ApiStatistics对象,用于存储从Redis中获取的API统计信息。
  • 合并与写入文件:
    • 将从Redis获取到的统计信息与本地的统计信息合并,然后将统计结果写入到api-stat.txt文件中,用于后续分析。
. . . . . . . . .
Terabox Video Player