大数据与流处理 / 系统课时
计算实时交易指标
计算实时交易指标
本课把原始交易事件转换为可直接监控的分钟级指标:按商户和支付渠道分组,统计交易金额、订单量、成功订单数与成功率,并保留窗口起止时间,便于后续写入指标库或驱动告警。
学习目标
- 明确交易金额、订单量和成功率的统计口径
- 使用事件时间滚动窗口按业务维度聚合交易事件
- 验证窗口结果,并识别迟到数据与重复订单的影响
指标口径与计算链路
窗口聚合前先固定口径。交易金额通常统计进入分析范围的订单金额;订单量按唯一订单计数;成功率等于成功订单数除以有效订单数。退款、撤销是否冲减金额,必须由业务规则明确,不能只凭支付状态猜测。
组合 AggregateFunction 与 ProcessWindowFunction,可以在事件进入窗口时增量累加金额和数量,并在窗口触发时补充窗口起止时间,避免保存全部明细。该组合方式适合既要聚合效率又要窗口元数据的场景。:contentReference[oaicite:0]{index=0}

示例选择 merchantId + channel 作为分组键。生产项目还可加入币种、区域或产品类型,但维度越多,键数量和状态规模越大,应只保留真正用于分析和告警的维度。
完整实现
下面的 Java 程序使用固定测试事件、5 秒乱序容忍和 1 分钟滚动窗口。它可放入已经配置 Flink DataStream 依赖的 Maven 项目中运行,依赖版本应与实际集群保持一致。
java
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.api.common.functions.AggregateFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.api.java.tuple.Tuple3;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.windowing.ProcessWindowFunction;
import org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows;
import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
import org.apache.flink.util.Collector;
import java.io.Serializable;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.time.Duration;
import java.time.Instant;
import java.util.Locale;
public class RealtimeTradeMetricsJob {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
DataStream<TradeEvent> events = env.fromElements(
event("T1001", "M001", "ALIPAY", "100.00", "SUCCESS",
"2026-08-04T04:00:05Z"),
event("T1002", "M001", "ALIPAY", "50.00", "FAILED",
"2026-08-04T04:00:25Z"),
event("T1003", "M002", "CARD", "200.00", "SUCCESS",
"2026-08-04T04:00:40Z"),
event("T1004", "M001", "ALIPAY", "80.00", "FAILED",
"2026-08-04T04:01:10Z")
);
WatermarkStrategy<TradeEvent> watermarks =
WatermarkStrategy.<TradeEvent>forBoundedOutOfOrderness(
Duration.ofSeconds(5))
.withTimestampAssigner((event, timestamp) -> event.eventTime);
events.assignTimestampsAndWatermarks(watermarks)
.keyBy(event -> Tuple2.of(event.merchantId, event.channel))
.window(TumblingEventTimeWindows.of(Duration.ofMinutes(1)))
.aggregate(new MetricAggregate(), new MetricWindowResult())
.print();
env.execute("realtime-trade-metrics");
}
private static TradeEvent event(
String orderId, String merchantId, String channel,
String amount, String payStatus, String eventTime) {
return new TradeEvent(orderId, merchantId, channel,
new BigDecimal(amount), payStatus,
Instant.parse(eventTime).toEpochMilli());
}
public static class MetricAggregate implements AggregateFunction<
TradeEvent, Tuple3<BigDecimal, Long, Long>,
Tuple3<BigDecimal, Long, Long>> {
@Override
public Tuple3<BigDecimal, Long, Long> createAccumulator() {
return Tuple3.of(BigDecimal.ZERO, 0L, 0L);
}
@Override
public Tuple3<BigDecimal, Long, Long> add(
TradeEvent event, Tuple3<BigDecimal, Long, Long> acc) {
acc.f0 = acc.f0.add(event.amount);
acc.f1++;
if ("SUCCESS".equals(event.payStatus)) {
acc.f2++;
}
return acc;
}
@Override
public Tuple3<BigDecimal, Long, Long> getResult(
Tuple3<BigDecimal, Long, Long> acc) {
return acc;
}
@Override
public Tuple3<BigDecimal, Long, Long> merge(
Tuple3<BigDecimal, Long, Long> left,
Tuple3<BigDecimal, Long, Long> right) {
return Tuple3.of(
left.f0.add(right.f0),
left.f1 + right.f1,
left.f2 + right.f2);
}
}
public static class MetricWindowResult extends ProcessWindowFunction<
Tuple3<BigDecimal, Long, Long>, String,
Tuple2<String, String>, TimeWindow> {
@Override
public void process(
Tuple2<String, String> key,
Context context,
Iterable<Tuple3<BigDecimal, Long, Long>> values,
Collector<String> out) {
Tuple3<BigDecimal, Long, Long> acc = values.iterator().next();
BigDecimal rate = BigDecimal.valueOf(acc.f2 * 100.0 / acc.f1)
.setScale(2, RoundingMode.HALF_UP);
out.collect(String.format(Locale.ROOT,
"merchant=%s, channel=%s, window=[%s,%s), "
+ "amount=%s, orders=%d, successRate=%s%%",
key.f0, key.f1,
Instant.ofEpochMilli(context.window().getStart()),
Instant.ofEpochMilli(context.window().getEnd()),
acc.f0.setScale(2, RoundingMode.HALF_UP),
acc.f1, rate));
}
}
public static class TradeEvent implements Serializable {
public String orderId;
public String merchantId;
public String channel;
public BigDecimal amount;
public String payStatus;
public long eventTime;
public TradeEvent() {
}
public TradeEvent(
String orderId, String merchantId, String channel,
BigDecimal amount, String payStatus, long eventTime) {
this.orderId = orderId;
this.merchantId = merchantId;
this.channel = channel;
this.amount = amount;
this.payStatus = payStatus;
this.eventTime = eventTime;
}
}
}预期输出如下,首个窗口内不同分组的打印顺序可能不同:
text
merchant=M001, channel=ALIPAY, window=[2026-08-04T04:00:00Z,2026-08-04T04:01:00Z), amount=150.00, orders=2, successRate=50.00%
merchant=M002, channel=CARD, window=[2026-08-04T04:00:00Z,2026-08-04T04:01:00Z), amount=200.00, orders=1, successRate=100.00%
merchant=M001, channel=ALIPAY, window=[2026-08-04T04:01:00Z,2026-08-04T04:02:00Z), amount=80.00, orders=1, successRate=0.00%验证与生产边界
先核对同一窗口内的原始订单,再验证金额、总数和成功数,最后计算成功率,避免只看最终百分比。时间展示为 UTC;接入报表前应统一转换为业务时区,但窗口计算仍应使用明确的事件时间。
生产流必须先按 orderId 去重,否则 Kafka 重放或上游重试会同时放大金额和订单量。超过允许迟到时间的事件应进入侧输出流或补偿任务;退款、撤销等反向事件则需要定义冲正口径,而不是直接当作失败订单。
按自己的节奏,确认真正理解。