大数据与流处理 / 系统课时
缓存与限流设计
缓存与限流设计
异步查询能隐藏外部存储的等待时间,但也可能把瞬时流量直接放大到数据库或 HTTP 服务。本课给出一套可落地的保护链路:先查本地缓存,再合并同键请求,用并发许可限制外部调用,最后在超时或过载时返回可识别的降级结果。
学习目标
- 设计缓存、请求合并、并发限制与降级的执行顺序
- 使用
AsyncFunction避免阻塞 Flink 算子线程 - 根据命中率、等待队列和外部服务容量调整关键参数
保护链路怎么组织
一次异步查询建议按“缓存命中 → 同键请求合并 → 获取并发许可 → 调用外部服务 → 写入缓存”的顺序执行。缓存减少重复访问,请求合并避免缓存失效瞬间产生击穿,并发许可则把单个并行子任务能够施加给外部系统的压力限制在可控范围内。

本地缓存只在当前 TaskManager 进程和算子实例内有效。它适合缓存体积小、允许短暂陈旧的数据,不应被当作跨节点一致性缓存。并行度变化、任务重启或重新调度后,缓存内容都会重新建立。
完整示例
下面的程序把保护逻辑放进 AsyncFunction。Semaphore 将每个并行子任务的外部并发限制为 2;inFlight 合并相同键的并发请求;超过 50 毫秒仍拿不到许可时返回降级值。示例使用 orderedWait 便于观察输入顺序,生产环境不要求保序时可改为 unorderedWait 提升吞吐。
import org.apache.flink.streaming.api.datastream.AsyncDataStream;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.async.AsyncFunction;
import org.apache.flink.streaming.api.functions.async.ResultFuture;
import java.io.Serializable;
import java.util.Collections;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
public class AsyncLookupProtectionDemo {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
DataStream<String> keys = env.fromElements("A", "B", "C", "A");
DataStream<String> enriched = AsyncDataStream.orderedWait(
keys,
new ProtectedLookup(),
2,
TimeUnit.SECONDS,
4
);
enriched.print();
env.execute("async-lookup-protection-demo");
}
public static final class ProtectedLookup
implements AsyncFunction<String, String> {
private final Map<String, CacheEntry> cache =
new ConcurrentHashMap<>();
private final Map<String, CompletableFuture<String>> inFlight =
new ConcurrentHashMap<>();
private final Semaphore permits = new Semaphore(2);
private transient ExecutorService executor;
@Override
public void asyncInvoke(
String key,
ResultFuture<String> resultFuture) {
CacheEntry hit = cache.get(key);
if (hit != null && hit.expireAt > System.currentTimeMillis()) {
resultFuture.complete(
Collections.singletonList(key + "|CACHE|" + hit.value));
return;
}
CompletableFuture<String> shared = inFlight.computeIfAbsent(
key,
currentKey -> CompletableFuture
.supplyAsync(
() -> loadWithLimit(currentKey),
executor())
.whenComplete(
(value, error) ->
inFlight.remove(currentKey))
);
shared.whenComplete((value, error) -> {
if (error != null) {
resultFuture.complete(
Collections.singletonList(
key + "|DEGRADED|UNKNOWN"));
return;
}
resultFuture.complete(Collections.singletonList(value));
});
}
private String loadWithLimit(String key) {
boolean acquired = false;
try {
acquired = permits.tryAcquire(50, TimeUnit.MILLISECONDS);
if (!acquired) {
return key + "|DEGRADED|UNKNOWN";
}
Thread.sleep(200);
String value = "profile-" + key;
cache.put(
key,
new CacheEntry(
value,
System.currentTimeMillis() + 10_000));
return key + "|REMOTE|" + value;
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return key + "|DEGRADED|UNKNOWN";
} finally {
if (acquired) {
permits.release();
}
}
}
private synchronized ExecutorService executor() {
if (executor == null) {
executor = Executors.newFixedThreadPool(4, runnable -> {
Thread thread = new Thread(runnable);
thread.setDaemon(true);
return thread;
});
}
return executor;
}
}
public static final class CacheEntry implements Serializable {
private final String value;
private final long expireAt;
private CacheEntry(String value, long expireAt) {
this.value = value;
this.expireAt = expireAt;
}
}
}预期输出
A|REMOTE|profile-A
B|REMOTE|profile-B
C|DEGRADED|UNKNOWN
A|REMOTE|profile-A第二个 A 与第一个 A 共享同一个在途请求,因此不会再次访问外部服务。实际控制台通常还会带并行子任务编号;若改用 unorderedWait,记录顺序可能不同,但每条输入仍只完成一次异步回调。
参数与边界
AsyncDataStream 的 capacity 控制每个异步算子并行实例最多保留多少个未完成请求;示例中的 Semaphore 则进一步控制真正进入外部服务的请求数。前者用于触发 Flink 背压,后者用于保护依赖系统,两者不应混为一个参数。
缓存 TTL 应短于业务允许的数据陈旧时间。降级值必须带明确状态,例如 DEGRADED,避免下游把兜底数据当成真实查询结果。对支付、库存扣减等强一致写操作,不应直接使用本地缓存或静默降级。
生产验收
上线前至少观察缓存命中率、同键合并次数、许可获取失败数、异步超时数、外部调用延迟和算子背压。若外部服务已满载,盲目提高线程数或 capacity 只会扩大排队和超时,应优先降低并发上限、缩短等待时间或增加可控的隔离资源。