当前位置：首页 > news >正文

[.NET] Aspire Dashboard: 云原生可观测性

news 2025/11/9 12:04:42

Aspire Dashboard 遥测数据采集机制详解

概述

.NET Aspire Dashboard是一个专门为分布式应用程序设计的可观测性平台，它通过OpenTelemetry协议(OTLP)采集和展示应用程序的日志(Logs)、指标(Metrics)和追踪(Traces)三大类遥测数据。Dashboard同时支持gRPC和HTTP两种传输协议，为应用程序提供了灵活的数据上报方式。

整体架构

Aspire Dashboard的遥测数据采集架构主要包含以下几个核心组件：

graph TBsubgraph "应用程序/服务"A1["日志生成"]A2["指标生成"] A3["追踪生成"]A1 --> OTELA2 --> OTELA3 --> OTELOTEL["OpenTelemetry SDK<br/>(OTLP Export)"]endOTEL -->|HTTP/gRPC| DASHBOARDsubgraph DASHBOARD ["Aspire Dashboard"]subgraph ENDPOINTS ["OTLP 接收端点"]subgraph HTTP_EP ["HTTP 端点"]H1["/v1/logs"]H2["/v1/traces"]H3["/v1/metrics"]endsubgraph GRPC_EP ["gRPC 端点"]G1["OtlpGrpcLogsService"]G2["OtlpGrpcTraceService"]G3["OtlpGrpcMetricsService"]endendENDPOINTS --> STORAGEsubgraph STORAGE ["数据处理与存储层"]subgraph REPO ["TelemetryRepository"]subgraph STORES ["存储组件"]S1["Logs 存储<br/>CircularBuffer<OtlpLogEntry>"]S2["Metrics 存储<br/>Resource Based Storage"]S3["Traces 存储<br/>CircularBuffer<OtlpTrace>"]endendendSTORAGE --> UIsubgraph UI ["Dashboard UI"]U1["日志查看器"]U2["指标图表"]U3["追踪分析器"]endendstyle A1 fill:#e1f5festyle A2 fill:#e8f5e8style A3 fill:#fff3e0style OTEL fill:#f3e5f5style S1 fill:#e1f5festyle S2 fill:#e8f5e8style S3 fill:#fff3e0

核心组件详解

1. OTLP接收端点

Dashboard提供两种接收遥测数据的方式：

HTTP端点 (`OtlpHttpEndpointsBuilder`)

路径: /v1/logs, /v1/traces, /v1/metrics
支持格式: Protocol Buffers (application/x-protobuf)
CORS支持: 可配置跨域资源共享
认证: 支持API Key认证

// HTTP端点映射示例
group.MapPost("logs", static (MessageBindable<ExportLogsServiceRequest> request, OtlpLogsService service) =>
{if (request.Message == null){return Results.Empty;}return OtlpResult.Response(service.Export(request.Message));
});

gRPC端点

服务: OtlpGrpcLogsService, OtlpGrpcTraceService, OtlpGrpcMetricsService
协议: OpenTelemetry标准gRPC服务
性能: 更高效的二进制传输

[Authorize(Policy = OtlpAuthorization.PolicyName)]
public class OtlpGrpcLogsService : LogsService.LogsServiceBase
{public override Task<ExportLogsServiceResponse> Export(ExportLogsServiceRequest request, ServerCallContext context){return Task.FromResult(_logsService.Export(request));}
}

2. 数据处理服务

日志处理 (`OtlpLogsService`)

public ExportLogsServiceResponse Export(ExportLogsServiceRequest request)
{var addContext = new AddContext();_telemetryRepository.AddLogs(addContext, request.ResourceLogs);return new ExportLogsServiceResponse{PartialSuccess = new ExportLogsPartialSuccess{RejectedLogRecords = addContext.FailureCount}};
}

追踪处理 (`OtlpTraceService`)

public ExportTraceServiceResponse Export(ExportTraceServiceRequest request)
{var addContext = new AddContext();_telemetryRepository.AddTraces(addContext, request.ResourceSpans);return new ExportTraceServiceResponse{PartialSuccess = new ExportTracePartialSuccess{RejectedSpans = addContext.FailureCount}};
}

指标处理 (`OtlpMetricsService`)

public ExportMetricsServiceResponse Export(ExportMetricsServiceRequest request)
{var addContext = new AddContext();_telemetryRepository.AddMetrics(addContext, request.ResourceMetrics);return new ExportMetricsServiceResponse{PartialSuccess = new ExportMetricsPartialSuccess{RejectedDataPoints = addContext.FailureCount}};
}

3. 数据存储层 (`TelemetryRepository`)

TelemetryRepository是Dashboard的核心数据管理组件，负责：

存储结构

日志存储: CircularBuffer<OtlpLogEntry> - 循环缓冲区，FIFO方式管理
追踪存储: CircularBuffer<OtlpTrace> - 支持容量管理和自动清理
指标存储: 基于Resource的存储模式
资源管理: ConcurrentDictionary<ResourceKey, OtlpResource>

数据容量管理

public sealed class TelemetryLimitOptions
{public int MaxLogCount { get; set; } = 10_000;        // 最大日志条数public int MaxTraceCount { get; set; } = 10_000;      // 最大追踪条数  public int MaxMetricsCount { get; set; } = 50_000;    // 最大指标点数public int MaxAttributeCount { get; set; } = 128;     // 最大属性数量public int MaxAttributeLength { get; set; } = int.MaxValue; // 最大属性长度public int MaxSpanEventCount { get; set; } = int.MaxValue;  // 最大Span事件数
}

数据插入机制

日志插入：支持乱序插入和时间戳排序

public void AddLogsCore(AddContext context, OtlpResourceView resourceView, RepeatedField<ScopeLogs> scopeLogs)
{_logsLock.EnterWriteLock();try{foreach (var record in sl.LogRecords){var logEntry = new OtlpLogEntry(record, resourceView, scope, _otlpContext);// 基于时间戳插入到正确位置var added = false;for (var i = _logs.Count - 1; i >= 0; i--){if (logEntry.TimeStamp > _logs[i].TimeStamp){_logs.Insert(i + 1, logEntry);added = true;break;}}if (!added){_logs.Insert(0, logEntry);}}}finally{_logsLock.ExitWriteLock();}
}

4. 实时订阅机制

Dashboard使用发布-订阅模式实现实时数据更新：

// 订阅管理
private readonly List<Subscription> _resourceSubscriptions = new();
private readonly List<Subscription> _logSubscriptions = new();
private readonly List<Subscription> _metricsSubscriptions = new();
private readonly List<Subscription> _tracesSubscriptions = new();// 触发订阅更新
private void RaiseSubscriptionChanged(List<Subscription> subscriptions)
{lock (_lock){foreach (var subscription in subscriptions){subscription.TryExecute();}}
}

5. 暂停管理 (`PauseManager`)

支持暂停数据采集功能，避免在调试期间数据过载：

public void AddLogs(AddContext context, RepeatedField<ResourceLogs> resourceLogs)
{if (_pauseManager.AreStructuredLogsPaused(out _)){_logger.LogTrace("{Count} incoming structured log(s) ignored because of an active pause.", resourceLogs.Count);return;}// ... 处理日志
}

6. 组件关系图

graph TDsubgraph "Dashboard Web Application"DWA[DashboardWebApplication] --> OTLP_HTTP[OtlpHttpEndpointsBuilder]DWA --> OTLP_GRPC[gRPC Services]DWA --> CONFIG[Configuration]subgraph "OTLP Services"OTLP_HTTP --> LS[OtlpLogsService]OTLP_HTTP --> TS[OtlpTraceService]OTLP_HTTP --> MS[OtlpMetricsService]OTLP_GRPC --> GLS[OtlpGrpcLogsService]OTLP_GRPC --> GTS[OtlpGrpcTraceService]OTLP_GRPC --> GMS[OtlpGrpcMetricsService]GLS --> LSGTS --> TSGMS --> MSendsubgraph "Data Layer"LS --> TR[TelemetryRepository]TS --> TRMS --> TRTR --> PM[PauseManager]TR --> CB1[CircularBuffer<Logs>]TR --> CB2[CircularBuffer<Traces>]TR --> RM[Resource Manager]TR --> SUB[Subscription System]endsubgraph "UI Components"SUB --> LV[Log Viewer]SUB --> MV[Metrics Viewer]SUB --> TV[Trace Viewer]endendsubgraph "Host Integration"RP[ResourcePublisher] --> DC[DashboardClient]DC --> TRendstyle DWA fill:#f9f,stroke:#333,stroke-width:3pxstyle TR fill:#bbf,stroke:#333,stroke-width:2pxstyle SUB fill:#bfb,stroke:#333,stroke-width:2px

配置与端点

端点配置 (`OtlpOptions`)

public sealed class OtlpOptions
{public string? PrimaryApiKey { get; set; }          // 主API密钥public string? SecondaryApiKey { get; set; }        // 备用API密钥  public OtlpAuthMode? AuthMode { get; set; }         // 认证模式public string? GrpcEndpointUrl { get; set; }        // gRPC端点URLpublic string? HttpEndpointUrl { get; set; }        // HTTP端点URLpublic OtlpCors Cors { get; set; } = new();         // CORS配置
}

CORS配置

public sealed class OtlpCors
{public string? AllowedOrigins { get; set; }   // 允许的来源域名public string? AllowedHeaders { get; set; }   // 允许的请求头[MemberNotNullWhen(true, nameof(AllowedOrigins))]public bool IsCorsEnabled => !string.IsNullOrEmpty(AllowedOrigins);
}

应用程序配置

Dashboard在DashboardWebApplication中进行完整的服务配置：

// 注册OTLP服务
builder.Services.AddSingleton<TelemetryRepository>();
builder.Services.AddTransient<OtlpLogsService>();
builder.Services.AddTransient<OtlpTraceService>();
builder.Services.AddTransient<OtlpMetricsService>();// 配置gRPC
builder.Services.AddGrpc();// 映射端点
_app.MapHttpOtlpApi(dashboardOptions.Otlp);      // HTTP端点
_app.MapGrpcService<OtlpGrpcMetricsService>();   // gRPC指标服务
_app.MapGrpcService<OtlpGrpcTraceService>();     // gRPC追踪服务  
_app.MapGrpcService<OtlpGrpcLogsService>();      // gRPC日志服务

与Aspire宿主的集成

资源发布 (`ResourcePublisher`)

Dashboard通过ResourcePublisher与Aspire宿主通信，获取应用程序资源信息：

internal sealed class ResourcePublisher
{private readonly Dictionary<string, SourceAndResourceSnapshot> _snapshot = [];private ImmutableHashSet<Channel<ResourceSnapshotChange>> _outgoingChannels = [];// 集成资源变更并广播给订阅者internal async ValueTask IntegrateAsync(IResource source, ResourceSnapshot snapshot, ResourceSnapshotChangeType changeType){lock (_syncLock){switch (changeType){case ResourceSnapshotChangeType.Upsert:_snapshot[snapshot.Name] = new SourceAndResourceSnapshot(source, snapshot);break;case ResourceSnapshotChangeType.Delete:_snapshot.Remove(snapshot.Name);break;}}// 通知所有订阅者foreach (var channel in channels){await channel.Writer.WriteAsync(new(changeType, snapshot), cancellationToken);}}
}

Dashboard客户端 (`DashboardClient`)

实现与资源服务的gRPC通信：

internal sealed class DashboardClient : IDashboardClient
{private readonly Dictionary<string, ResourceViewModel> _resourceByName = new();private readonly GrpcChannel? _channel;private Aspire.DashboardService.Proto.V1.DashboardService.DashboardServiceClient? _client;// 订阅资源变更public async IAsyncEnumerable<IReadOnlyList<ResourceViewModelChange>> SubscribeResourcesAsync(){// 通过gRPC流式接收资源更新}
}

数据流转过程

1. 数据接收流程

sequenceDiagramparticipant App as 应用程序participant OTLP as OTLP Exporterparticipant HTTP as HTTP端点participant GRPC as gRPC端点participant Service as 数据处理服务participant Repo as TelemetryRepositoryparticipant Storage as 存储层App->>OTLP: 生成遥测数据OTLP->>HTTP: HTTP/JSON 请求OTLP->>GRPC: gRPC 调用HTTP->>Service: OtlpLogsService.Export()GRPC->>Service: OtlpGrpcLogsService.Export()Service->>Repo: AddLogs/AddTraces/AddMetricsRepo->>Storage: 写入CircularBufferStorage-->>Repo: 存储确认Repo-->>Service: 处理结果Service-->>HTTP: ExportResponseService-->>GRPC: ExportResponse

2. 数据查询流程

sequenceDiagramparticipant UI as Dashboard UIparticipant Sub as 订阅系统participant Repo as TelemetryRepositoryparticipant Filter as 数据过滤器participant Storage as 存储层UI->>Sub: 订阅数据更新Sub->>Repo: 注册订阅回调loop 实时数据查询UI->>Repo: 查询遥测数据Repo->>Filter: 应用过滤条件Filter->>Storage: 读取数据Storage-->>Filter: 返回数据Filter-->>Repo: 过滤后数据Repo-->>UI: 返回结果endNote over Storage: 数据变更时Storage->>Repo: 触发变更事件Repo->>Sub: 通知订阅者Sub->>UI: 实时推送更新

3. 实时更新机制

flowchart LRA[数据变更] --> B[触发订阅]B --> C[推送到UI组件]C --> D[前端实时刷新]subgraph 订阅管理E[ResourceSubscriptions]F[LogSubscriptions]G[MetricsSubscriptions]H[TracesSubscriptions]endB --> EB --> FB --> GB --> H