大数据与流处理 / 系统课时

计算实时交易指标

计算实时交易指标

本课把原始交易事件转换为可直接监控的分钟级指标:按商户和支付渠道分组,统计交易金额、订单量、成功订单数与成功率,并保留窗口起止时间,便于后续写入指标库或驱动告警。

学习目标

  • 明确交易金额、订单量和成功率的统计口径
  • 使用事件时间滚动窗口按业务维度聚合交易事件
  • 验证窗口结果,并识别迟到数据与重复订单的影响

指标口径与计算链路

窗口聚合前先固定口径。交易金额通常统计进入分析范围的订单金额;订单量按唯一订单计数;成功率等于成功订单数除以有效订单数。退款、撤销是否冲减金额,必须由业务规则明确,不能只凭支付状态猜测。

组合 AggregateFunction 与 ProcessWindowFunction,可以在事件进入窗口时增量累加金额和数量,并在窗口触发时补充窗口起止时间,避免保存全部明细。该组合方式适合既要聚合效率又要窗口元数据的场景。:contentReference[oaicite:0]{index=0}

交易指标窗口计算链路

示例选择 merchantId + channel 作为分组键。生产项目还可加入币种、区域或产品类型,但维度越多,键数量和状态规模越大,应只保留真正用于分析和告警的维度。

完整实现

下面的 Java 程序使用固定测试事件、5 秒乱序容忍和 1 分钟滚动窗口。它可放入已经配置 Flink DataStream 依赖的 Maven 项目中运行,依赖版本应与实际集群保持一致。

java
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.api.common.functions.AggregateFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.api.java.tuple.Tuple3;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.windowing.ProcessWindowFunction;
import org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows;
import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
import org.apache.flink.util.Collector;

import java.io.Serializable;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.time.Duration;
import java.time.Instant;
import java.util.Locale;

public class RealtimeTradeMetricsJob {

    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env =
                StreamExecutionEnvironment.getExecutionEnvironment();
        env.setParallelism(1);

        DataStream<TradeEvent> events = env.fromElements(
                event("T1001", "M001", "ALIPAY", "100.00", "SUCCESS",
                        "2026-08-04T04:00:05Z"),
                event("T1002", "M001", "ALIPAY", "50.00", "FAILED",
                        "2026-08-04T04:00:25Z"),
                event("T1003", "M002", "CARD", "200.00", "SUCCESS",
                        "2026-08-04T04:00:40Z"),
                event("T1004", "M001", "ALIPAY", "80.00", "FAILED",
                        "2026-08-04T04:01:10Z")
        );

        WatermarkStrategy<TradeEvent> watermarks =
                WatermarkStrategy.<TradeEvent>forBoundedOutOfOrderness(
                                Duration.ofSeconds(5))
                        .withTimestampAssigner((event, timestamp) -> event.eventTime);

        events.assignTimestampsAndWatermarks(watermarks)
                .keyBy(event -> Tuple2.of(event.merchantId, event.channel))
                .window(TumblingEventTimeWindows.of(Duration.ofMinutes(1)))
                .aggregate(new MetricAggregate(), new MetricWindowResult())
                .print();

        env.execute("realtime-trade-metrics");
    }

    private static TradeEvent event(
            String orderId, String merchantId, String channel,
            String amount, String payStatus, String eventTime) {
        return new TradeEvent(orderId, merchantId, channel,
                new BigDecimal(amount), payStatus,
                Instant.parse(eventTime).toEpochMilli());
    }

    public static class MetricAggregate implements AggregateFunction<
            TradeEvent, Tuple3<BigDecimal, Long, Long>,
            Tuple3<BigDecimal, Long, Long>> {

        @Override
        public Tuple3<BigDecimal, Long, Long> createAccumulator() {
            return Tuple3.of(BigDecimal.ZERO, 0L, 0L);
        }

        @Override
        public Tuple3<BigDecimal, Long, Long> add(
                TradeEvent event, Tuple3<BigDecimal, Long, Long> acc) {
            acc.f0 = acc.f0.add(event.amount);
            acc.f1++;
            if ("SUCCESS".equals(event.payStatus)) {
                acc.f2++;
            }
            return acc;
        }

        @Override
        public Tuple3<BigDecimal, Long, Long> getResult(
                Tuple3<BigDecimal, Long, Long> acc) {
            return acc;
        }

        @Override
        public Tuple3<BigDecimal, Long, Long> merge(
                Tuple3<BigDecimal, Long, Long> left,
                Tuple3<BigDecimal, Long, Long> right) {
            return Tuple3.of(
                    left.f0.add(right.f0),
                    left.f1 + right.f1,
                    left.f2 + right.f2);
        }
    }

    public static class MetricWindowResult extends ProcessWindowFunction<
            Tuple3<BigDecimal, Long, Long>, String,
            Tuple2<String, String>, TimeWindow> {

        @Override
        public void process(
                Tuple2<String, String> key,
                Context context,
                Iterable<Tuple3<BigDecimal, Long, Long>> values,
                Collector<String> out) {
            Tuple3<BigDecimal, Long, Long> acc = values.iterator().next();
            BigDecimal rate = BigDecimal.valueOf(acc.f2 * 100.0 / acc.f1)
                    .setScale(2, RoundingMode.HALF_UP);

            out.collect(String.format(Locale.ROOT,
                    "merchant=%s, channel=%s, window=[%s,%s), "
                            + "amount=%s, orders=%d, successRate=%s%%",
                    key.f0, key.f1,
                    Instant.ofEpochMilli(context.window().getStart()),
                    Instant.ofEpochMilli(context.window().getEnd()),
                    acc.f0.setScale(2, RoundingMode.HALF_UP),
                    acc.f1, rate));
        }
    }

    public static class TradeEvent implements Serializable {
        public String orderId;
        public String merchantId;
        public String channel;
        public BigDecimal amount;
        public String payStatus;
        public long eventTime;

        public TradeEvent() {
        }

        public TradeEvent(
                String orderId, String merchantId, String channel,
                BigDecimal amount, String payStatus, long eventTime) {
            this.orderId = orderId;
            this.merchantId = merchantId;
            this.channel = channel;
            this.amount = amount;
            this.payStatus = payStatus;
            this.eventTime = eventTime;
        }
    }
}

预期输出如下,首个窗口内不同分组的打印顺序可能不同:

text
merchant=M001, channel=ALIPAY, window=[2026-08-04T04:00:00Z,2026-08-04T04:01:00Z), amount=150.00, orders=2, successRate=50.00%
merchant=M002, channel=CARD, window=[2026-08-04T04:00:00Z,2026-08-04T04:01:00Z), amount=200.00, orders=1, successRate=100.00%
merchant=M001, channel=ALIPAY, window=[2026-08-04T04:01:00Z,2026-08-04T04:02:00Z), amount=80.00, orders=1, successRate=0.00%

验证与生产边界

先核对同一窗口内的原始订单,再验证金额、总数和成功数,最后计算成功率,避免只看最终百分比。时间展示为 UTC;接入报表前应统一转换为业务时区,但窗口计算仍应使用明确的事件时间。

生产流必须先按 orderId 去重,否则 Kafka 重放或上游重试会同时放大金额和订单量。超过允许迟到时间的事件应进入侧输出流或补偿任务;退款、撤销等反向事件则需要定义冲正口径,而不是直接当作失败订单。

按自己的节奏,确认真正理解。

目录与阅读设置

阅读设置

17

记录一个想法

计算实时交易指标

0 / 10000 · 笔记保存到账号。未提交的修改会在关闭、离开或刷新后丢失。

登录哪都通