
  1. Ŀ                             
                                   
  UCCP  Intel OWT (Open WebRTC Toolkit) 죬һֲʽʵʱý崦ƽ̨
  -  Apache 2.0 license + OWT  SFU+MCU ϼܹ                                                                                                                       
  - Node.js Ʋ + C++ ý + AI           
  - ԭ" AI ǿ + UCCP = Unified Cloud Content Production+ PUGC ֱ"
                                                                                                                                                                            
  2. Ŀ¼ṹ                                  
                                                                                                                                                                            
  uccp_server/                                                                     
   source/                                                                                                                                                               
      agent/               Node.js ҵ̳أ10+  agent                                                                                                          
      core/                C++ ý崦棨owt_base                                                                                                                
      management_api/      REST API (3000 ˿)                                                                                                                        
      management_console/  Web ̨ (3300)                                                                                                                           
      portal/              Socket.IO  (8080)                                                                                                                       
      cluster_manager/     Ⱥ                                                                                                                                    
      delegation/          PUGC/UCCP ί                                                                                                                          
      sip_portal/          SIP                                                                                                                                     
      process_monitor/                                                                                                                                         
      rabbitMQManagement/  MQ                                                                                                                                      
      data_access/         Mongo ʲ                                                                                                                                
      common/                                                                                                                                                  
   build/libdeps/           FFmpeg/OpenCV/CUDA  3rd-party .so                                                                                                         
   scripts/  doc/  cert/  docker/  services/  test/                                                                                                                      
                                                                                                                                                                            
  3. ջ                                                                                                                                                                 
                                                                                                                                                                            
  Щ                                                                                
                                                                                                                                                                   
  ੤                                                                                
   ҵ/  Node.js + Express + Socket.IO + amqplib + mongoose + log4js                                                                                                
  ੤                                                                                
   ý   C++ + OWT + FFmpeg + NVENC/NVDEC + OpenCV+CUDA                                                                                                             
  ੤                                                                                
   ͨ       RabbitMQ ( RPC) + ZeroMQ ( IPC) + Socket.IO () + HTTP REST                                                                                 
  ੤                                                                                
   洢       MongoDB//û+ RabbitMQϢ                                
  ੤                                                                                
   AI         dlopen ص plugin .so + Triton/Fssrv ⲿ                     
  ة                                                                                
                                                                                                                                                                            
  4. ̬                                                                                                                                                               
                                                                                                                                                                            
  ̷ֲʽһ uccp  10+ ̣ȫ RabbitMQ                                                                                                  
                                                                                   
  Щ                                                                                                                  
                              ˿/ɫ                                     
  ੤                                                                                                                  
   management_api      3000 REST                                                
  ੤                                                                                                                  
   management_console  3300 Web UI                                              
  ੤                                                                                                                  
   portal              8080 Socket.IO                                       
  ੤                                                                                                                  
   cluster_manager     ȣ MQ                                            
  ੤                                                                                                                  
   conference agent    鷿                                             
  ੤                                                                                                                  
   video agent         Ƶ/ת                                                                                                                                     
  ੤                                                                                                                  
   audio agent         Ƶ                                                                                                                                          
  ੤                                                                                                                  
   webrtc agent        WebRTC                                               
  ੤                                                                                                                  
   streaming agent     ֱ/                          
  ੤                                                                                                                  
   recording agent     ¼                                                     
  ੤                                                                                                                  
   sip agent           SIP                                                  
  ੤                                                                                                                  
   process_monitor     Ź                                                   
  ة                                                                                                                  
                                                                                   
  5. ģ嵥ǹؼֻǱɽһǣ                                                                                                                       
                                                                                   
  ֣                                                                                                                                                                
                                                                                   
  ý崦                                                                                                                                                              
  - ƵVideoMixer ·ϳ                                          
  - Ƶת루VideoTranscoder ʽת                                                                                                                              
  - ƵAudioMixer                                                         
  - WebRTC 루SFU                                                                                                                                                      
  - ¼Ʒ                                                                       
  - SIP                                                                                                                                                                 
                                                                                   
  AI ǿ source/core/owt_base/AI*.cpp                                                                                                                              
  - ?? 켣AILightBasketballStar.cpp ǹĵľһ               
  - ?.켣.AILighFootballStar.cpp                     .                                                                                                        
  - ??..켣AILightFHockeyStar.cpp                                       
  - ?? OCR ȷʶAILightStarOcrInference.cpp                                                                                                                          
  - ?? ֲ루AIadvertsInference.cpp                                                                                                                                   
  - ?? ͨ AI ܣAIInference.h                                                                                                                                    
                                                                                                                                                                            
  ҵ                                                                                                                                                                
  - PUGC ֱpugcLiveTaskController.js                                                                                                                              
  - PUGC VOD pugcVodTaskController.js                                                                                                                               
  - UCCP uccpTaskController.js                                                                                                                                
  - UCCP uccp_mix/                                                                                                                                                  
  - ˿ƣdigitalHumanController.js                                                                                                                                 
  - ⳡplaneController.js                                                                                                                                          
  - Ͽ⣨corpusController.js                                                                                                                                           
                                                                                                                                                                            
  6. ܹͼ                                                                                                                                                               
                                                                                                                                                                            
                                                                                                         
                           û                                                                                                                                     
    Web Console    REST API    Socket.IO Portal    SIP ͻ                                                                                                          
     :3300          :3000          :8080                                                                                                                               
  ЩЩЩЩ                                                                                                       
                                                                                                                                                                        
          ةЩة                                                                                                                    
                                   HTTP/WS/SIP                                                                                                                             
                                                                                                                                                                           
                                                                                                         
                      Ʋ㣨Node.js                                                                                                                                   
     cluster_manager ?? delegation ?? process_monitor                                                                                                                 
                                                                                                                                                                       
           ة                                                                                                                        
                                                                                                                                                                         
  ੤                                                                                                       
                                     RabbitMQ (RPC + ¼)                                                                                                                 
                                                                                                                                                                           
                                                                                                         
               ҵ Agent ̳أNode.jsˮƽչ                                                                                                                   
                                                                                                                 
    conference   video      audio     webrtc    sip                                                                                                             
      agent      agent      agent      agent    agent                                                                                                           
                                                                                                                 
                                                                                                                          
    analytics  streaming  recording    quic      ...                                                                                                              
      agent      agent ?      agent                                                                                                                      
                                                                                                                              
                                                                                                                                                              
  Щ                                                                                                       
                                     N-API  C++ addon                                                                                                                  
                                                                                                                                                                           
                                                                                                         
                ý崦㣨C++OWT ܣ                                                                                                                               
                                                                                                                                                                          
                    Ĵ                                                                                                                                  
                                                                                                                                    
    LiveStrm  ?    FFNvFrame     ?   FFNvFrame                                                                                                              
      In       װ  Decoder        GPU    Encoder                                                                                                                
    (FFmpeg)           (NVDEC)        NV12   (NVENC)                                                                                                                
             Щ         Щ                                                                                                              
                                                                                                                                                                        
                                  AI                                                                                                                                
                                                                                                                                              
                         FrameProcesser ? SeiGenera-                                                                                                              
                                        AI JSON  tor                                                                                                                  
                              ֡   SEI NALU                                                                                                             
                           AI*Star                                                                                                                        
                           (ͬ)                                                                                                                                     
                           Щ                                                                                                                                   
                         ੤                                                                                                                                 
                                  dlopen                                                                                                                                 
  ੤                                                                                                      
                                                                                                                                                                           
                                                                                                         
             AI Plugin 㣨.so ̬⣬ҵأ                                                                                                                       
    libpluginSDKBasketballTraj.so   libpluginSDKFootballTraj.so                                                                                                           
    libpluginSDKHockeyTraj.so       libpluginSDKOCR.so                                                                                                                    
                                                                                                                                                                        
            gRPC                      gRPC                                                                                                                              
  ੤੤                                                                                                       
                                                                                                                                                                          
                                                                                                                                                                          
                                                                                                         
     ⲿTriton (2403/2401/2402/2064) + Fssrv +                                                                                                                 
     ball_detect gRPC :6099 (migu_basketball_yolo)                                                                                                                        
                                                                                                         
                                                                                                                                                                            
                                                                                                         
                        ʩ                                                                                                                                          
     RabbitMQ ?  RPC     MongoDB  /Ự/û״̬                                                                                                            
                                                                                                         
 ǻڴ뿼 uccp                                                                                                                             
                                                                                                                                                                            
  1. ںͻỰ                                                                                                                                                         
                                                                                                                                                                            
  Щ                           
                                                                             λ                                                                                      
  ੤                           
               source/agent/index.jsNode.js ػ̣                                                                                                           
  ੤                           
   C++ װ      source/agent/addons/avstreamLib/AVStreamInWrap.cc:309 ?? new owt_base::LiveStreamIn(...)                                                            
  ੤                           
   Ựڹ  VideoFrameTranscoderImpl (VideoFrameTranscoderImpl.h:51)setInput()  addOutput()  ...  removeOutput()  unsetInput()                            
  ੤                           
                 source/agent/video/agent.toml + JS  Options ͸ C++                                                                 
  ة                           
                                                                                   
  2.  &                                                                                                                                                             
                                                                                                                                                                            
  - LiveStreamIn.cpp:1143 avformat_open_input()LiveStreamIn.cpp:2639 av_read_frame() ??  FFmpeg                                                                 
  - ̣߳LiveStreamIn.cpp:1029 boost::thread(&LiveStreamIn::receiveLoop, this)                                                                                         
  -                                                                                                                                                         
    - FFNvFrameDecoder ?? NVDEC FFmpeg h264_cuvid + AV_PIX_FMT_CUDA    
    - NvcuvidFrameDecoder ?? ֱ NVIDIA nvcuvid SDKѱ #if 0 ע͵                                                                                                   
    - л㣺VideoFrameDecoderAdapter.cpp:65-76                                   
  - NvFrameں cv::cuda::GpuMat dataNV12 GPU ڴ                                                                                                                
                                                                                   
  3. AI                                                                                                                                                             
                                                                                   
  - ͬFFNvFrameDecoder::onFrame  FrameProcesser::onFrame  AILightBasketballStar::processûж AI ̣߳                                                    
  - AI 룺boost::shared_ptr<NvLightStarFrame>ں NvVideoFrame->dataGpuMat                                                                                         
  - Plugin У                                                                                                                                                       
    - AILightBasketballStar.cpp:38-39 dlopen("./lib/libpluginSDKBasketballTraj.so")                                                                                         
    - AILightBasketballStar.cpp:45 dlsym("createMiguPluginClientInstance")                                                                                                  
    - AILightBasketballStar.cpp:381 client->process(ai_input, processOutput)                                                                                                
  - AI ݣNvFrame::addAIMetaData(json)  JSON ֡ϣ֡ߵ                                                                                        
                                                                                                                                                                            
  4.  & SEI ע                                                                                                                                                        
                                                                                   
  - FFNvFrameEncoderFFNvFrameEncoder.h:41 NvFrameGPU NV12                                                                                             
  - AI ԪȡFFNvFrameEncoder.cpp:1018 inFrame->getAIMetaData(aiMetaData)                                                                                             
  - SEI ע㣺FFNvFrameEncoder.cpp:1071-1079 JSON ƴ outFrame.additionalInfo.video.seiData                                                                          
  - SEI UUID"mg-sei-user-data" (SeiGenerator.cpp:23)                                                                                                                      
  - JSON װAILightBasketballStar.cpp:407 {"name":"HighLightStar","parameters":<plugin_out>,"frameIndex":N}                                                              
                                                                                                                                                                            
  5.                                                                                                                                                                
                                                                                                                                                                            
  - LiveStreamOut ̳ AVStreamOutavformat_write_header() / av_interleaved_write_frame()                                                                                  
  - Э飺RTMP / UDP / SRT                                                         
  - ʱAVStreamOutWrap.cc:331 requireDelayPlay / delayPlayTime MediaFrameQueue Ч                                                                            
                                                                                                                                                                            
  6. ߳ģͣÿỰ                                                                                                                                                     
                                                                                                                                                                            
  Щ                                                                                                                    
              ߳                     ְ                                                                                                                              
  ੤                                                                                                                    
   LiveStreamIn::m_thread       read + װ                               
  ੤                                                                                                                    
   FFNvFrameDecoder ڲ       NVDEC                                                                                                                                 
  ੤                                                                                                                    
   ޶ AI ̣߳          AI ڽ߳ͬ                                                                                                                     
  ੤                                                                                                                    
   FFNvFrameEncoder::m_thread  GPU  + SEI                                   
  ੤                                                                                                                    
   LiveStreamOut ڲ           write                                       
  ة                                                                                                                    
                                                                                                                                                                            
  ͬԭboost::mutex + boost::condition_variableУFramePacketBuffer / JitterBuffer˶֡                                                                  
                                                                                                                                                                            
  7. ͼ                                                                                                                                                           
                                                                                   
                                                                                                                                      
                           Node.js agent (index.js)                              
                           ͸ Options  C++                                                                                                                             
                         Щ                    
                                         new LiveStreamIn                                                                                                                  
                                        ?                                          
                                                                                                                
       LiveStreamIn  (uccp_server/.../LiveStreamIn.cpp)                          
                                                                                                                                                                          
       avformat_open_input  [:1143]       RTMP/UDP/SRT/RTSP                                                                                                              
                                                                                                                                                                         
             ?                                                                                                                                                            
       av_read_frame  [:2639]             AVPacket                                                                                                                       
                                                                                                                                                                         
             ?                                                                                                                                                            
       deliverVideoFrame  [:3352]         FramePacketBuffer                                                                                                              
                                                                                                                                                                         
             ?                                                                                                                                                            
       deliverFrame(onFrame)  [:3437]     FrameDestination                                                                                                               
     ੤                                                                                                           
                                                                                                                                                                           
              ?                                                                                                                                                             
                                                                                                                
       VideoFrameDecoderAdapter::onFrame  [:135]                                                                                                                          
                                                                                                                                                                         
             ?                                                                                                                                                            
       FFNvFrameDecoder    h264_cuvid + AV_PIX_FMT_CUDA                                                                                                                  
       (NvcuvidFrameDecoder ѱ #if 0 ע͵)                                                                                                                            
                                                                                                                                                                         
             ?  GPU ڴ NV12                                                                                                                                             
       NvFrame { cv::cuda::GpuMat data; list<string> AIMeta; }                                                                                                            
     ੤                                                                                                           
               FrameDestination::onFrame(NvFrame)                                                                                                                          
              ?                                                                                                                                                             
                                                                                                                
       FrameProcesser::onFrame                                                                                                                                            
                                                                                                                                                                         
             ?  SendNvFrame                                                                                                                                               
       AILightBasketballStar::process  [:214]                                                                                                                             
                                                                                                                                                                         
                ai_input.image.addr[0] = frame->data.data        GPU ֱ                                                                                               
                ai_input.image.pitch[0]= frame->data.step                                                                                                                
                                                                                                                                                                         
             ?                                                                                                                                                            
       dlopen libpluginSDKBasketballTraj.so                                                                                                                               
       client->process(ai_input, output)  [:381]                                                                                                                          
                                                                                                                                                                         
             ?  JSON                                                                                                                                                      
       NvFrame::addAIMetaData(json)   Ԫݸ֡                                                                                                                     
     ੤                                                                                                           
                                                                                                                                                                           
              ?                                                                                                                                                             
                                                                                                                
       FFNvFrameEncoder::onFrame  [:655]                                                                                                                                  
                                                                                                                                                                         
             ?  NvFrame  AVFrame (CUDA frame)                                                                                                                            
       Encode  (h264_nvenc)                                                                                                                                               
                                                                                                                                                                         
             ?  AVPacket                                                                                                                                                  
       getAIMetaData  [:1018]     ȡ֡Ϲҵ JSON                                                                                                                      
                                                                                                                                                                         
             ?                                                                                                                                                            
       SeiGenerator::generateSEI  (UUID="mg-sei-user-data")                                                                                                               
                                                                                                                                                                         
             ?                                                                                                                                                            
       outFrame.additionalInfo.video.seiData  [:1079]                                                                                                                     
                                                                                                                                                                         
             ?                                                                                                                                                            
       deliverFrame  [:1099]                                                                                                                                              
     ੤                                                                                                           
                                                                                                                                                                           
              ?                                                                                                                                                             
                                                                                                                
       LiveStreamOut / AVStreamOut                                                                                                                                        
                                                                                                                                                                          
       av_interleaved_write_frame()       RTMP/UDP/SRT                                                                                                                   
                                                                                                                                                                          
       delayPlayTime / requireDelayPlay  MediaFrameQueue Ч                                                                                                           
                                                                                                                
                                                                                                                                                                            