人工智能大模型MLOpsLLMOps模型推理服务云原生后端【免费下载链接】seldon-coreAn MLOps framework to package, deploy, monitor and manage thousands of production machine learning models项目地址https://gitcode.com/gh_mirrors/se/seldon-core点击查看免费下载Seldon Core 2 的 Pipeline 之间并非彼此孤立一条 Pipeline 的 outputs 可以作为另一条 Pipeline 的输入、触发器甚至可以精确到某个 step 或某个 tensor。本文基于仓库示例文档 pipeline-to-pipeline.md 和 samples 目录下的配套 YAML完整演示 5 种跨 Pipeline 连接方式单源消费、双源汇聚、数据触发、step 级引用、同模型 step 复用并给出每一步的可复制命令、预期输出以及从 operator 与 scheduler 源码侧对这些字段的实现佐证读完即可在生产集群seldon-mesh命名空间中搭建可追溯的多 Pipeline 数据流。一、准备环境与模型1. 前置条件已在命名空间seldon-mesh中完成 Seldon Core 2 安装在已下载 samples 示例文件的目录中执行后续命令获取以 Istio 方式运行的 Seldon Core 2 实例的 Ingress IPISTIO_INGRESS$(kubectl get svc seldon-mesh -n seldon-mesh -o jsonpath{.status.loadBalancer.ingress[0].ip}) echo Seldon Core 2: http://$ISTIO_INGRESS记录输出的 IP 地址后文所有命令中的INGRESS_IP均替换为该服务网格入口 IP。2. 使用的模型所有用例共用的两个 Model 均指向同一个 Triton TensorFlow 示例模型gs://seldon-models/triton/simple其计算逻辑为OUTPUT0 INPUT0 INPUT1OUTPUT1 INPUT0 - INPUT1因此当INPUT0与INPUT1都取[1,2,...,16]时OUTPUT0 [2,4,...,32]OUTPUT1全 0。这个确定性的加减法模型使跨 Pipeline 的张量流转可以用数值精确验证。两个 Model 的 YAML对应 tfsimple1.yaml 与 tfsimple2.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: tfsimple1 spec: storageUri: gs://seldon-models/triton/simple requirements: - tensorflow memory: 100Ki --- apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: tfsimple2 spec: storageUri: gs://seldon-models/triton/simple requirements: - tensorflow memory: 100Ki创建并等待就绪kubectl create -f ./models/tfsimple1.yaml -n seldon-mesh kubectl create -f ./models/tfsimple2.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s model --all -n seldon-mesh预期输出model.mlops.seldon.io/tfsimple1 created model.mlops.seldon.io/tfsimple2 created model.mlops.seldon.io/tfsimple1 condition met model.mlops.seldon.io/tfsimple2 condition met二、基础 Pipelinetfsimple首先部署一条不含外部输入的单 step Pipelinetfsimple对应 tfsimple.yamlcat ./pipelines/tfsimple.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple spec: steps: - name: tfsimple1 output: steps: - tfsimple1kubectl create -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh预期输出pipeline.mlops.seldon.io/tfsimple created pipeline.mlops.seldon.io/tfsimple condition met用 curl 或 seldon-cli 发起推理。注意两个关键请求头Host: seldon-mesh.inference.seldon用于服务网格路由Seldon-Model: tfsimple.pipeline用于把请求路由到名为tfsimple的Pipeline模型请求则使用name.model后缀curl -k http://INGRESS_IP:80/v2/models/tfsimple/infer \ -H Host: seldon-mesh.inference.seldon \ -H Content-Type: application/json \ -H Seldon-Model: tfsimple.pipeline \ -d { inputs: [ { name: INPUT0, datatype: INT32, shape: [1,16], data: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] }, { name: INPUT1, datatype: INT32, shape: [1,16], data: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] } ] } | jq -M .等价的 CLI 方式seldon pipeline infer tfsimple --inference-host INGRESS_IP:80\ {inputs:[{name:INPUT0,data:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],datatype:INT32,shape:[1,16]},{name:INPUT1,data:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],datatype:INT32,shape:[1,16]}]} | jq -M .响应中OUTPUT0为[2,4,6,...,32]OUTPUT1为 16 个 0shape 均为[1,16]、datatype 为INT32{ model_name: , outputs: [ { data: [2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32], name: OUTPUT0, shape: [1,16], datatype: INT32 }, { data: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0], name: OUTPUT1, shape: [1,16], datatype: INT32 } ] }这条 Pipeline 的tfsimple.outputs从此可作为下游 Pipeline 的数据源进入下面 5 个用例。三、用例 1一条 Pipeline 消费另一条 Pipeline 的 outputsexternalInputs tensorMap1. 部署 tfsimple-extended对应 tfsimple-extended.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-extended spec: input: externalInputs: - tfsimple.outputs tensorMap: tfsimple.outputs.OUTPUT0: INPUT0 tfsimple.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2各字段含义与 pipeline_types.go 中PipelineInput结构一致externalInputs声明本 Pipeline 的外部数据来源。引用格式为上游pipeline.outputs整个输出的所有 tensor也支持上游pipeline.outputs.tensor精确引用单个 tensortensorMap张量重命名映射键为来源.tensor值为下游 step 期望的输入名。这里把上游的OUTPUT0/OUTPUT1改名为INPUT0/INPUT1正好匹配 Triton simple 模型的入参joinWindowMs可选多输入 join 时等待消息到达的窗口毫秒joinType默认inner还支持outer、any源码见 pipeline_types.go 中JoinTypeInner/JoinTypeOuter/JoinTypeAny。kubectl create -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-meshpipeline.mlops.seldon.io/tfsimple-extended created pipeline.mlops.seldon.io/tfsimple-extended condition met2. 发起推理并用 inspect 验证数据流请求仍打向最上游的tfsimple。第二次请求携带x-request-id: test-id便于在 inspect 输出中按请求 ID 过滤curl -k http://INGRESS_IP:80/v2/models/tfsimple/infer \ -H Host: seldon-mesh.inference.seldon \ -H Seldon-Model: tfsimple.pipeline \ -H x-request-id: test-id \ -H Content-Type: application/json \ -d {...} | jq -M . # 或 seldon pipeline infer tfsimple --header x-request-idtest-id --inference-host INGRESS_IP:80 \ {inputs:[{name:INPUT0,data:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],datatype:INT32,shape:[1,16]},{name:INPUT1,data:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],datatype:INT32,shape:[1,16]}]}{...}为第一节相同的请求体。响应仍是tfsimple自身的输出OUTPUT0[2,...,32]、OUTPUT1[0,...,0]真正体现跨 Pipeline 流转的是seldon pipeline inspectseldon pipeline inspect tfsimpleseldon.default.model.tfsimple1.inputs test-id {inputs:[{INPUT0: 1..16}, {INPUT1: 1..16}]} seldon.default.model.tfsimple1.outputs test-id {OUTPUT0: 2,4,...,32; OUTPUT1: 0...0} seldon.default.pipeline.tfsimple.inputs test-id {INPUT0: 1..16; INPUT1: 1..16} seldon.default.pipeline.tfsimple.outputs test-id {OUTPUT0: 2,4,...,32; OUTPUT1: 0...0}seldon pipeline inspect tfsimple-extended关键行原文档完整输出见 pipeline-to-pipeline.mdseldon.default.model.tfsimple2.inputs test-id {inputs:[{name:INPUT0,...,intContents:[2,4,...,32]},{name:INPUT1,...,intContents:[0,0,...,0]}]} seldon.default.model.tfsimple2.outputs test-id {OUTPUT0: 2,4,...,32; OUTPUT1: 2,4,...,32} seldon.default.pipeline.tfsimple-extended.inputs test-id {INPUT0: 2,4,...,32; INPUT1: 0...0} seldon.default.pipeline.tfsimple-extended.outputs test-id {OUTPUT0: 2,4,...,32; OUTPUT1: 2,4,...,32}可以清楚看到tfsimple2收到的INPUT0正是上游tfsimple.outputs.OUTPUT0即[2,4,...,32]、INPUT1是OUTPUT1全 0因此其OUTPUT0 20... [2,...,32]、OUTPUT1 2-0... [2,...,32]。inspect 输出的 topic 前缀seldon.default.pipeline.name.inputs/outputs正是数据平面为每条 Pipeline 建立 Kafka topic 的证据见第六节。seldon pipeline inspect支持pipelineName.(inputs|outputs)、pipelineName.stepName、pipelineName.stepName.(inputs|outputs)、.tensorName等表达式常用 flag 有--offset从第几条消息开始读默认 1 即最近一条、--request-id按请求 ID 过滤、--format raw|json、--verbose显示pipeline[...]、x-seldon-route、traceparent等头部细节详见 seldon_pipeline_inspect.md。3. 清理kubectl delete -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple1.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple2.yaml -n seldon-mesh四、用例 2一条 Pipeline 同时消费两条 Pipeline 的指定 tensor本用例建立三条 Pipeline两条相同的tfsimple-extended/tfsimple-extended2均消费tfsimple.outputs以及汇聚 Pipelinetfsimple-combined。对应 tfsimple-extended2.yaml 与 tfsimple-combined.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-extended2 spec: input: externalInputs: - tfsimple.outputs tensorMap: tfsimple.outputs.OUTPUT0: INPUT0 tfsimple.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2 --- apiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-combined spec: input: externalInputs: - tfsimple-extended.outputs.OUTPUT0 - tfsimple-extended2.outputs.OUTPUT1 tensorMap: tfsimple-extended.outputs.OUTPUT0: INPUT0 tfsimple-extended2.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2combined的externalInputs展示了 tensor 级精确引用只取tfsimple-extended的OUTPUT0作为INPUT0只取tfsimple-extended2的OUTPUT1作为INPUT1。由于两个 extended Pipeline 输入相同OUTPUT0[2,...,32]、OUTPUT1[2,...,32]所以combined的INPUT0与INPUT1相等得到OUTPUT0[4,8,...,64]、OUTPUT1全 0。kubectl create -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-combined.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh对tfsimple发起带x-request-id: test-id2的推理请求体同前然后验证seldon pipeline inspect tfsimple seldon pipeline inspect tfsimple-extended --offset 2 --verbose seldon pipeline inspect tfsimple-extended2 --offset 2 seldon pipeline inspect tfsimple-combined--offset 2用于跳过部署/早期探测消息只看业务流量--verbose会额外打印pipeline[tfsimple-extended]、x-seldon-route[:tfsimple1_1: :tfsimple2_1:]、traceparent[00-...]等头部可用于跨 Pipeline 的分布式追踪。tfsimple-combined的 inspect 输出确认了汇聚逻辑seldon.default.model.tfsimple2.inputs test-id {INPUT0: 2,4,...,32; INPUT1: 2,4,...,32} seldon.default.model.tfsimple2.outputs test-id {OUTPUT0: 4,8,...,64; OUTPUT1: 0...0} seldon.default.pipeline.tfsimple-combined.inputs test-id {INPUT0: 2,4,...,32; INPUT1: 2,4,...,32} seldon.default.pipeline.tfsimple-combined.outputs test-id {OUTPUT0: 4,8,...,64; OUTPUT1: 0...0}清理kubectl delete -f ./pipelines/tfsimple-combined.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple1.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple2.yaml -n seldon-mesh五、用例 3一条 Pipeline 的 outputs 作数据、另一条的 outputs 作触发externalTriggers对应 tfsimple-combined-trigger.yaml。tfsimple-extended与tfsimple-extended2的部署同用例 2区别在汇聚 PipelineapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-combined-trigger spec: input: externalInputs: - tfsimple-extended.outputs externalTriggers: - tfsimple-extended2.outputs tensorMap: tfsimple-extended.outputs.OUTPUT0: INPUT0 tfsimple-extended.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2语义区别tfsimple-extended2.outputs声明为externalTriggers后其消息只提供到达信号、不贡献张量——tfsimple2的计算输入仅来自tfsimple-extended的OUTPUT0/OUTPUT1经 tensorMap 改名而tfsimple-extended2的输出到达则充当闸门。这适合以 A 数据为主、等 B 侧事件到来才执行的场景。triggersJoinType控制触发消息的 join 语义默认inner。kubectl create -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-combined-trigger.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh带x-request-id: test-id3向tfsimple发起推理后验证seldon pipeline inspect tfsimple seldon pipeline inspect tfsimple-extended --offset 2 seldon pipeline inspect tfsimple-extended2 --offset 2 seldon pipeline inspect tfsimple-combined-triggertfsimple-combined-trigger的 inspect 输出确认其 inputs 的INPUT0与INPUT1都来自tfsimple-extended均为[2,...,32]与[2,...,32]tfsimple-extended2仅触发最终seldon.default.model.tfsimple2.inputs test-id3 {INPUT0: 2,4,...,32; INPUT1: 2,4,...,32} seldon.default.model.tfsimple2.outputs test-id3 {OUTPUT0: 4,8,...,64; OUTPUT1: 0...0}清理命令与用例 2 相同把tfsimple-combined换成tfsimple-combined-trigger。六、用例 4 与 5直接引用另一条 Pipeline 内特定 step 的 outputs跨 Pipeline 引用不必停留在 Pipeline 级还可以下钻到 step 级语法为pipeline.step.stepName.outputs并可继续追加.tensorName。1. 引用单条 Pipeline 的单个 steptfsimple-extended-step对应 tfsimple-extended-step.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-extended-step spec: input: externalInputs: - tfsimple.step.tfsimple1.outputs tensorMap: tfsimple.step.tfsimple1.outputs.OUTPUT0: INPUT0 tfsimple.step.tfsimple1.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2这里引用的是tfsimple内部 steptfsimple1的输出而非 Pipeline 出口。当上游 Pipeline 有多个 step 时这种写法让下游精确挂接到中间结果上。kubectl create -f ./pipelines/tfsimple-extended-step.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh向tfsimple推理后seldon pipeline inspect tfsimple --verbose seldon pipeline inspect tfsimple-extended-steptfsimple-extended-step的输入为{INPUT0: 2,4,...,32; INPUT1: 0...0}输出{OUTPUT0: 2,4,...,32; OUTPUT1: 2,4,...,32}与用例 1 的数值一致——step 级引用在本例中与 Pipeline 级引用等价因为 step 即唯一出口其价值在多 step 图景中体现。kubectl delete -f ./pipelines/tfsimple-extended-step.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple1.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple2.yaml -n seldon-mesh2. 从两条 Pipeline 的同一模型 step 拉取 tensortfsimple-combined-step对应 tfsimple-combined-step.yaml。部署tfsimple-extended、tfsimple-extended2同用例 2后apiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-combined-step spec: input: externalInputs: - tfsimple-extended.step.tfsimple2.outputs.OUTPUT0 - tfsimple-extended2.step.tfsimple2.outputs.OUTPUT0 tensorMap: tfsimple-extended.step.tfsimple2.outputs.OUTPUT0: INPUT0 tfsimple-extended2.step.tfsimple2.outputs.OUTPUT0: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2这条 Pipeline 从两条上游 Pipeline 中同一个 steptfsimple2的同一 tensorOUTPUT0各取一份作为INPUT0/INPUT1。两条上游的tfsimple2.OUTPUT0均为[2,...,32]故combined-step输出OUTPUT0[4,8,...,64]、OUTPUT1全 0kubectl create -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-combined-step.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh向tfsimple推理后依次 inspect 四条 Pipelinetfsimple、tfsimple-extended、tfsimple-extended2、tfsimple-combined-steptfsimple-combined-step输出seldon.default.model.tfsimple2.inputs ... {INPUT0: 2,4,...,32; INPUT1: 2,4,...,32} seldon.default.model.tfsimple2.outputs ... {OUTPUT0: 4,8,...,64; OUTPUT1: 0...0}清理kubectl delete -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-combined-step.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple1.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple2.yaml -n seldon-mesh七、引用格式速查与源码级佐证1. externalInputs / externalTriggers 引用格式引用形式含义pipeline.outputs消费该 Pipeline 出口的全部 tensor每个 tensor 需有对应 tensorMap 条目pipeline.outputs.tensor只消费该出口 tensorpipeline.step.stepName.outputs消费上游内部某 step 的全部输出pipeline.step.stepName.outputs.tensor消费上游内部某 step 的单个 tensorexternalTriggers: 上述任意形式同上引用但仅作为触发信号不参与张量计算tensorMap键为来源引用.tensor值为下游 step 的输入张量名来源引用与externalInputs条目前缀保持一致。2. CRD 定义到调度器的转换跨 Pipeline 能力的类型定义在 pipeline_types.goPipelineInput结构体包含ExternalInputs、ExternalTriggers、JoinWindowMs、JoinType默认inner可选outer/any、TriggersJoinType、TensorMap六个字段PipelineStep还有对应的Triggers、Batchsize/windowMs/rolling等字段说明触发与批处理语义在 step 级同样成立。同文件的AsSchedulerPipeline()方法L160-L272把 CRD 字段一一转换为 scheduler 的 protobufscheduler.Pipeline表示其中 join 类型映射为INNER/OUTER/ANY枚举。3. Operator 的调谐链路pipeline_controller.go 中的PipelineReconciler负责把 Pipeline 对象同步进数据平面Reconcile()调用Scheduler.LoadPipeline(ctx, pipeline, nil)通过 gRPC 把 Pipeline 加载进 schedulerhandleFinalizer()则在删除时先调用Scheduler.UnloadPipeline卸载从源码结构看这也是删除 Pipeline 时 Kafka topic 等数据平面资源得以回收的机制失败重试耗尽后才移除 finalizer。kubectl wait --for conditionready等待的Ready条件即由PipelineReady、PipelineGwReady、ModelsReady三个条件组共同决定见 pipeline_types.go。4. 数据平面如何承载跨 Pipeline 消息inspector 看到的 topic 前缀如seldon.default.pipeline.tfsimple-extended.inputs对应 topics.go 中的命名规则GetPipelineTopicInputs/GetPipelineTopicOutputs生成topicPrefix.namespace.pipeline.name.inputs|outputsGetModelOrPipelineTopicAndTensorL98 起按.拆分 step 引用pipeline.step.stepName...解析出 topic 与可选 tensor 名——这正是第六节--verbose输出里seldon.default.pipeline.*消息流的来源。seldon pipeline inspect即读取这些 topic因此它能按请求 ID 与 offset 回放每条 Pipeline 的输入/输出消息配合x-seldon-route记录实际经过的模型路由如[:tfsimple1_1: :tfsimple2_1:]与traceparent可完成跨 Pipeline 的请求级追踪。八、小结Seldon Core 2 用input.externalInputs/input.externalTriggerstensorMap三个字段即可把任意多条 Pipeline 组织成有向数据流图支持 Pipeline 级、step 级、tensor 级三种粒度触发器trigger与数据input语义分离实现事件驱动型 Pipeline所有跨 Pipeline 流量在 Kafka 上按prefix.namespace.pipeline.name.inputs|outputs组织 topicseldon pipeline inspect提供按--offset、--request-id、--verbose的审计与调试手段部署顺序上先建 Model 再建上游 Pipeline、最后建下游 Pipeline删除时反序先下游、后上游、最后 Model可避免引用悬空导致的 Ready 失败。赞分享人工智能大模型MLOpsLLMOps模型推理服务云原生后端【免费下载链接】seldon-coreAn MLOps framework to package, deploy, monitor and manage thousands of production machine learning models项目地址https://gitcode.com/gh_mirrors/se/seldon-core点击查看免费下载相关推荐Seldon Core v2 seldon pipeline inspect 实战深入 Pipeline 内部数据流进行请求级调试Seldon Core v2 seldon pipeline inspect 实战深入 Pipeline 内部数据流进行请求级调试 seldon pipeli人工智能大模型MLOpsLLMOps模型推理服务云原生后端Seldon Core 2 seldon pipeline unload 命令详解卸载 Pipeline 的完整用法与底层实现Seldon Core 2 seldon pipeline unload 命令详解卸载 Pipeline 的完整用法与底层实现 本文是一份围绕 Seldon人工智能大模型MLOpsLLMOps模型推理服务云原生后端Seldon Core 2 循环 Pipeline 实战用 allowCycles 与 maxStepRevisits 构建反馈回路Seldon Core 2 循环 Pipeline 实战用 allowCycles 与 maxStepRevisits 构建反馈回路 本文基于 docs gb人工智能大模型MLOpsLLMOps模型推理服务云原生后端创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 
长河君 · 2026/10/13 13:35:32 · 阅读 1.2k Seldon Core 2 Pipeline-to-Pipeline 实战：用 externalInputs、externalTriggers 与 tensorMap 打通跨 Pipeline 数据流
人工智能大模型MLOpsLLMOps模型推理服务云原生后端【免费下载链接】seldon-coreAn MLOps framework to package, deploy, monitor and manage thousands of production machine learning models项目地址https://gitcode.com/gh_mirrors/se/seldon-core点击查看免费下载Seldon Core 2 的 Pipeline 之间并非彼此孤立一条 Pipeline 的 outputs 可以作为另一条 Pipeline 的输入、触发器甚至可以精确到某个 step 或某个 tensor。本文基于仓库示例文档 pipeline-to-pipeline.md 和 samples 目录下的配套 YAML完整演示 5 种跨 Pipeline 连接方式单源消费、双源汇聚、数据触发、step 级引用、同模型 step 复用并给出每一步的可复制命令、预期输出以及从 operator 与 scheduler 源码侧对这些字段的实现佐证读完即可在生产集群seldon-mesh命名空间中搭建可追溯的多 Pipeline 数据流。一、准备环境与模型1. 前置条件已在命名空间seldon-mesh中完成 Seldon Core 2 安装在已下载 samples 示例文件的目录中执行后续命令获取以 Istio 方式运行的 Seldon Core 2 实例的 Ingress IPISTIO_INGRESS$(kubectl get svc seldon-mesh -n seldon-mesh -o jsonpath{.status.loadBalancer.ingress[0].ip}) echo Seldon Core 2: http://$ISTIO_INGRESS记录输出的 IP 地址后文所有命令中的INGRESS_IP均替换为该服务网格入口 IP。2. 使用的模型所有用例共用的两个 Model 均指向同一个 Triton TensorFlow 示例模型gs://seldon-models/triton/simple其计算逻辑为OUTPUT0 INPUT0 INPUT1OUTPUT1 INPUT0 - INPUT1因此当INPUT0与INPUT1都取[1,2,...,16]时OUTPUT0 [2,4,...,32]OUTPUT1全 0。这个确定性的加减法模型使跨 Pipeline 的张量流转可以用数值精确验证。两个 Model 的 YAML对应 tfsimple1.yaml 与 tfsimple2.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: tfsimple1 spec: storageUri: gs://seldon-models/triton/simple requirements: - tensorflow memory: 100Ki --- apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: tfsimple2 spec: storageUri: gs://seldon-models/triton/simple requirements: - tensorflow memory: 100Ki创建并等待就绪kubectl create -f ./models/tfsimple1.yaml -n seldon-mesh kubectl create -f ./models/tfsimple2.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s model --all -n seldon-mesh预期输出model.mlops.seldon.io/tfsimple1 created model.mlops.seldon.io/tfsimple2 created model.mlops.seldon.io/tfsimple1 condition met model.mlops.seldon.io/tfsimple2 condition met二、基础 Pipelinetfsimple首先部署一条不含外部输入的单 step Pipelinetfsimple对应 tfsimple.yamlcat ./pipelines/tfsimple.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple spec: steps: - name: tfsimple1 output: steps: - tfsimple1kubectl create -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh预期输出pipeline.mlops.seldon.io/tfsimple created pipeline.mlops.seldon.io/tfsimple condition met用 curl 或 seldon-cli 发起推理。注意两个关键请求头Host: seldon-mesh.inference.seldon用于服务网格路由Seldon-Model: tfsimple.pipeline用于把请求路由到名为tfsimple的Pipeline模型请求则使用name.model后缀curl -k http://INGRESS_IP:80/v2/models/tfsimple/infer \ -H Host: seldon-mesh.inference.seldon \ -H Content-Type: application/json \ -H Seldon-Model: tfsimple.pipeline \ -d { inputs: [ { name: INPUT0, datatype: INT32, shape: [1,16], data: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] }, { name: INPUT1, datatype: INT32, shape: [1,16], data: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16] } ] } | jq -M .等价的 CLI 方式seldon pipeline infer tfsimple --inference-host INGRESS_IP:80\ {inputs:[{name:INPUT0,data:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],datatype:INT32,shape:[1,16]},{name:INPUT1,data:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],datatype:INT32,shape:[1,16]}]} | jq -M .响应中OUTPUT0为[2,4,6,...,32]OUTPUT1为 16 个 0shape 均为[1,16]、datatype 为INT32{ model_name: , outputs: [ { data: [2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32], name: OUTPUT0, shape: [1,16], datatype: INT32 }, { data: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0], name: OUTPUT1, shape: [1,16], datatype: INT32 } ] }这条 Pipeline 的tfsimple.outputs从此可作为下游 Pipeline 的数据源进入下面 5 个用例。三、用例 1一条 Pipeline 消费另一条 Pipeline 的 outputsexternalInputs tensorMap1. 部署 tfsimple-extended对应 tfsimple-extended.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-extended spec: input: externalInputs: - tfsimple.outputs tensorMap: tfsimple.outputs.OUTPUT0: INPUT0 tfsimple.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2各字段含义与 pipeline_types.go 中PipelineInput结构一致externalInputs声明本 Pipeline 的外部数据来源。引用格式为上游pipeline.outputs整个输出的所有 tensor也支持上游pipeline.outputs.tensor精确引用单个 tensortensorMap张量重命名映射键为来源.tensor值为下游 step 期望的输入名。这里把上游的OUTPUT0/OUTPUT1改名为INPUT0/INPUT1正好匹配 Triton simple 模型的入参joinWindowMs可选多输入 join 时等待消息到达的窗口毫秒joinType默认inner还支持outer、any源码见 pipeline_types.go 中JoinTypeInner/JoinTypeOuter/JoinTypeAny。kubectl create -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-meshpipeline.mlops.seldon.io/tfsimple-extended created pipeline.mlops.seldon.io/tfsimple-extended condition met2. 发起推理并用 inspect 验证数据流请求仍打向最上游的tfsimple。第二次请求携带x-request-id: test-id便于在 inspect 输出中按请求 ID 过滤curl -k http://INGRESS_IP:80/v2/models/tfsimple/infer \ -H Host: seldon-mesh.inference.seldon \ -H Seldon-Model: tfsimple.pipeline \ -H x-request-id: test-id \ -H Content-Type: application/json \ -d {...} | jq -M . # 或 seldon pipeline infer tfsimple --header x-request-idtest-id --inference-host INGRESS_IP:80 \ {inputs:[{name:INPUT0,data:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],datatype:INT32,shape:[1,16]},{name:INPUT1,data:[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],datatype:INT32,shape:[1,16]}]}{...}为第一节相同的请求体。响应仍是tfsimple自身的输出OUTPUT0[2,...,32]、OUTPUT1[0,...,0]真正体现跨 Pipeline 流转的是seldon pipeline inspectseldon pipeline inspect tfsimpleseldon.default.model.tfsimple1.inputs test-id {inputs:[{INPUT0: 1..16}, {INPUT1: 1..16}]} seldon.default.model.tfsimple1.outputs test-id {OUTPUT0: 2,4,...,32; OUTPUT1: 0...0} seldon.default.pipeline.tfsimple.inputs test-id {INPUT0: 1..16; INPUT1: 1..16} seldon.default.pipeline.tfsimple.outputs test-id {OUTPUT0: 2,4,...,32; OUTPUT1: 0...0}seldon pipeline inspect tfsimple-extended关键行原文档完整输出见 pipeline-to-pipeline.mdseldon.default.model.tfsimple2.inputs test-id {inputs:[{name:INPUT0,...,intContents:[2,4,...,32]},{name:INPUT1,...,intContents:[0,0,...,0]}]} seldon.default.model.tfsimple2.outputs test-id {OUTPUT0: 2,4,...,32; OUTPUT1: 2,4,...,32} seldon.default.pipeline.tfsimple-extended.inputs test-id {INPUT0: 2,4,...,32; INPUT1: 0...0} seldon.default.pipeline.tfsimple-extended.outputs test-id {OUTPUT0: 2,4,...,32; OUTPUT1: 2,4,...,32}可以清楚看到tfsimple2收到的INPUT0正是上游tfsimple.outputs.OUTPUT0即[2,4,...,32]、INPUT1是OUTPUT1全 0因此其OUTPUT0 20... [2,...,32]、OUTPUT1 2-0... [2,...,32]。inspect 输出的 topic 前缀seldon.default.pipeline.name.inputs/outputs正是数据平面为每条 Pipeline 建立 Kafka topic 的证据见第六节。seldon pipeline inspect支持pipelineName.(inputs|outputs)、pipelineName.stepName、pipelineName.stepName.(inputs|outputs)、.tensorName等表达式常用 flag 有--offset从第几条消息开始读默认 1 即最近一条、--request-id按请求 ID 过滤、--format raw|json、--verbose显示pipeline[...]、x-seldon-route、traceparent等头部细节详见 seldon_pipeline_inspect.md。3. 清理kubectl delete -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple1.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple2.yaml -n seldon-mesh四、用例 2一条 Pipeline 同时消费两条 Pipeline 的指定 tensor本用例建立三条 Pipeline两条相同的tfsimple-extended/tfsimple-extended2均消费tfsimple.outputs以及汇聚 Pipelinetfsimple-combined。对应 tfsimple-extended2.yaml 与 tfsimple-combined.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-extended2 spec: input: externalInputs: - tfsimple.outputs tensorMap: tfsimple.outputs.OUTPUT0: INPUT0 tfsimple.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2 --- apiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-combined spec: input: externalInputs: - tfsimple-extended.outputs.OUTPUT0 - tfsimple-extended2.outputs.OUTPUT1 tensorMap: tfsimple-extended.outputs.OUTPUT0: INPUT0 tfsimple-extended2.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2combined的externalInputs展示了 tensor 级精确引用只取tfsimple-extended的OUTPUT0作为INPUT0只取tfsimple-extended2的OUTPUT1作为INPUT1。由于两个 extended Pipeline 输入相同OUTPUT0[2,...,32]、OUTPUT1[2,...,32]所以combined的INPUT0与INPUT1相等得到OUTPUT0[4,8,...,64]、OUTPUT1全 0。kubectl create -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-combined.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh对tfsimple发起带x-request-id: test-id2的推理请求体同前然后验证seldon pipeline inspect tfsimple seldon pipeline inspect tfsimple-extended --offset 2 --verbose seldon pipeline inspect tfsimple-extended2 --offset 2 seldon pipeline inspect tfsimple-combined--offset 2用于跳过部署/早期探测消息只看业务流量--verbose会额外打印pipeline[tfsimple-extended]、x-seldon-route[:tfsimple1_1: :tfsimple2_1:]、traceparent[00-...]等头部可用于跨 Pipeline 的分布式追踪。tfsimple-combined的 inspect 输出确认了汇聚逻辑seldon.default.model.tfsimple2.inputs test-id {INPUT0: 2,4,...,32; INPUT1: 2,4,...,32} seldon.default.model.tfsimple2.outputs test-id {OUTPUT0: 4,8,...,64; OUTPUT1: 0...0} seldon.default.pipeline.tfsimple-combined.inputs test-id {INPUT0: 2,4,...,32; INPUT1: 2,4,...,32} seldon.default.pipeline.tfsimple-combined.outputs test-id {OUTPUT0: 4,8,...,64; OUTPUT1: 0...0}清理kubectl delete -f ./pipelines/tfsimple-combined.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple1.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple2.yaml -n seldon-mesh五、用例 3一条 Pipeline 的 outputs 作数据、另一条的 outputs 作触发externalTriggers对应 tfsimple-combined-trigger.yaml。tfsimple-extended与tfsimple-extended2的部署同用例 2区别在汇聚 PipelineapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-combined-trigger spec: input: externalInputs: - tfsimple-extended.outputs externalTriggers: - tfsimple-extended2.outputs tensorMap: tfsimple-extended.outputs.OUTPUT0: INPUT0 tfsimple-extended.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2语义区别tfsimple-extended2.outputs声明为externalTriggers后其消息只提供到达信号、不贡献张量——tfsimple2的计算输入仅来自tfsimple-extended的OUTPUT0/OUTPUT1经 tensorMap 改名而tfsimple-extended2的输出到达则充当闸门。这适合以 A 数据为主、等 B 侧事件到来才执行的场景。triggersJoinType控制触发消息的 join 语义默认inner。kubectl create -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-combined-trigger.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh带x-request-id: test-id3向tfsimple发起推理后验证seldon pipeline inspect tfsimple seldon pipeline inspect tfsimple-extended --offset 2 seldon pipeline inspect tfsimple-extended2 --offset 2 seldon pipeline inspect tfsimple-combined-triggertfsimple-combined-trigger的 inspect 输出确认其 inputs 的INPUT0与INPUT1都来自tfsimple-extended均为[2,...,32]与[2,...,32]tfsimple-extended2仅触发最终seldon.default.model.tfsimple2.inputs test-id3 {INPUT0: 2,4,...,32; INPUT1: 2,4,...,32} seldon.default.model.tfsimple2.outputs test-id3 {OUTPUT0: 4,8,...,64; OUTPUT1: 0...0}清理命令与用例 2 相同把tfsimple-combined换成tfsimple-combined-trigger。六、用例 4 与 5直接引用另一条 Pipeline 内特定 step 的 outputs跨 Pipeline 引用不必停留在 Pipeline 级还可以下钻到 step 级语法为pipeline.step.stepName.outputs并可继续追加.tensorName。1. 引用单条 Pipeline 的单个 steptfsimple-extended-step对应 tfsimple-extended-step.yamlapiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-extended-step spec: input: externalInputs: - tfsimple.step.tfsimple1.outputs tensorMap: tfsimple.step.tfsimple1.outputs.OUTPUT0: INPUT0 tfsimple.step.tfsimple1.outputs.OUTPUT1: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2这里引用的是tfsimple内部 steptfsimple1的输出而非 Pipeline 出口。当上游 Pipeline 有多个 step 时这种写法让下游精确挂接到中间结果上。kubectl create -f ./pipelines/tfsimple-extended-step.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh向tfsimple推理后seldon pipeline inspect tfsimple --verbose seldon pipeline inspect tfsimple-extended-steptfsimple-extended-step的输入为{INPUT0: 2,4,...,32; INPUT1: 0...0}输出{OUTPUT0: 2,4,...,32; OUTPUT1: 2,4,...,32}与用例 1 的数值一致——step 级引用在本例中与 Pipeline 级引用等价因为 step 即唯一出口其价值在多 step 图景中体现。kubectl delete -f ./pipelines/tfsimple-extended-step.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple1.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple2.yaml -n seldon-mesh2. 从两条 Pipeline 的同一模型 step 拉取 tensortfsimple-combined-step对应 tfsimple-combined-step.yaml。部署tfsimple-extended、tfsimple-extended2同用例 2后apiVersion: mlops.seldon.io/v1alpha1 kind: Pipeline metadata: name: tfsimple-combined-step spec: input: externalInputs: - tfsimple-extended.step.tfsimple2.outputs.OUTPUT0 - tfsimple-extended2.step.tfsimple2.outputs.OUTPUT0 tensorMap: tfsimple-extended.step.tfsimple2.outputs.OUTPUT0: INPUT0 tfsimple-extended2.step.tfsimple2.outputs.OUTPUT0: INPUT1 steps: - name: tfsimple2 output: steps: - tfsimple2这条 Pipeline 从两条上游 Pipeline 中同一个 steptfsimple2的同一 tensorOUTPUT0各取一份作为INPUT0/INPUT1。两条上游的tfsimple2.OUTPUT0均为[2,...,32]故combined-step输出OUTPUT0[4,8,...,64]、OUTPUT1全 0kubectl create -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl create -f ./pipelines/tfsimple-combined-step.yaml -n seldon-mesh kubectl wait --for conditionready --timeout300s pipeline --all -n seldon-mesh向tfsimple推理后依次 inspect 四条 Pipelinetfsimple、tfsimple-extended、tfsimple-extended2、tfsimple-combined-steptfsimple-combined-step输出seldon.default.model.tfsimple2.inputs ... {INPUT0: 2,4,...,32; INPUT1: 2,4,...,32} seldon.default.model.tfsimple2.outputs ... {OUTPUT0: 4,8,...,64; OUTPUT1: 0...0}清理kubectl delete -f ./pipelines/tfsimple-extended.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-extended2.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple-combined-step.yaml -n seldon-mesh kubectl delete -f ./pipelines/tfsimple.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple1.yaml -n seldon-mesh kubectl delete -f ./models/tfsimple2.yaml -n seldon-mesh七、引用格式速查与源码级佐证1. externalInputs / externalTriggers 引用格式引用形式含义pipeline.outputs消费该 Pipeline 出口的全部 tensor每个 tensor 需有对应 tensorMap 条目pipeline.outputs.tensor只消费该出口 tensorpipeline.step.stepName.outputs消费上游内部某 step 的全部输出pipeline.step.stepName.outputs.tensor消费上游内部某 step 的单个 tensorexternalTriggers: 上述任意形式同上引用但仅作为触发信号不参与张量计算tensorMap键为来源引用.tensor值为下游 step 的输入张量名来源引用与externalInputs条目前缀保持一致。2. CRD 定义到调度器的转换跨 Pipeline 能力的类型定义在 pipeline_types.goPipelineInput结构体包含ExternalInputs、ExternalTriggers、JoinWindowMs、JoinType默认inner可选outer/any、TriggersJoinType、TensorMap六个字段PipelineStep还有对应的Triggers、Batchsize/windowMs/rolling等字段说明触发与批处理语义在 step 级同样成立。同文件的AsSchedulerPipeline()方法L160-L272把 CRD 字段一一转换为 scheduler 的 protobufscheduler.Pipeline表示其中 join 类型映射为INNER/OUTER/ANY枚举。3. Operator 的调谐链路pipeline_controller.go 中的PipelineReconciler负责把 Pipeline 对象同步进数据平面Reconcile()调用Scheduler.LoadPipeline(ctx, pipeline, nil)通过 gRPC 把 Pipeline 加载进 schedulerhandleFinalizer()则在删除时先调用Scheduler.UnloadPipeline卸载从源码结构看这也是删除 Pipeline 时 Kafka topic 等数据平面资源得以回收的机制失败重试耗尽后才移除 finalizer。kubectl wait --for conditionready等待的Ready条件即由PipelineReady、PipelineGwReady、ModelsReady三个条件组共同决定见 pipeline_types.go。4. 数据平面如何承载跨 Pipeline 消息inspector 看到的 topic 前缀如seldon.default.pipeline.tfsimple-extended.inputs对应 topics.go 中的命名规则GetPipelineTopicInputs/GetPipelineTopicOutputs生成topicPrefix.namespace.pipeline.name.inputs|outputsGetModelOrPipelineTopicAndTensorL98 起按.拆分 step 引用pipeline.step.stepName...解析出 topic 与可选 tensor 名——这正是第六节--verbose输出里seldon.default.pipeline.*消息流的来源。seldon pipeline inspect即读取这些 topic因此它能按请求 ID 与 offset 回放每条 Pipeline 的输入/输出消息配合x-seldon-route记录实际经过的模型路由如[:tfsimple1_1: :tfsimple2_1:]与traceparent可完成跨 Pipeline 的请求级追踪。八、小结Seldon Core 2 用input.externalInputs/input.externalTriggerstensorMap三个字段即可把任意多条 Pipeline 组织成有向数据流图支持 Pipeline 级、step 级、tensor 级三种粒度触发器trigger与数据input语义分离实现事件驱动型 Pipeline所有跨 Pipeline 流量在 Kafka 上按prefix.namespace.pipeline.name.inputs|outputs组织 topicseldon pipeline inspect提供按--offset、--request-id、--verbose的审计与调试手段部署顺序上先建 Model 再建上游 Pipeline、最后建下游 Pipeline删除时反序先下游、后上游、最后 Model可避免引用悬空导致的 Ready 失败。赞分享人工智能大模型MLOpsLLMOps模型推理服务云原生后端【免费下载链接】seldon-coreAn MLOps framework to package, deploy, monitor and manage thousands of production machine learning models项目地址https://gitcode.com/gh_mirrors/se/seldon-core点击查看免费下载相关推荐Seldon Core v2 seldon pipeline inspect 实战深入 Pipeline 内部数据流进行请求级调试Seldon Core v2 seldon pipeline inspect 实战深入 Pipeline 内部数据流进行请求级调试 seldon pipeli人工智能大模型MLOpsLLMOps模型推理服务云原生后端Seldon Core 2 seldon pipeline unload 命令详解卸载 Pipeline 的完整用法与底层实现Seldon Core 2 seldon pipeline unload 命令详解卸载 Pipeline 的完整用法与底层实现 本文是一份围绕 Seldon人工智能大模型MLOpsLLMOps模型推理服务云原生后端Seldon Core 2 循环 Pipeline 实战用 allowCycles 与 maxStepRevisits 构建反馈回路Seldon Core 2 循环 Pipeline 实战用 allowCycles 与 maxStepRevisits 构建反馈回路 本文基于 docs gb人工智能大模型MLOpsLLMOps模型推理服务云原生后端创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考