如何去实现基于MPP解码+live555的RTSP服务端呢

准备

编译MPP

• 首先下载自己的MPP源代码（瑞芯微github有最新版），我为了攀登的程序，使用的是SDK中的mpp。
  
• 进入mpp源码mpp/build/linux/aarch64中修改，arm.linux.cross.cmake文件中的配置（我没有）
  
• 运行make-Makefiles.bash
  

./make-Makefiles.bash
-- cmake version 3.5.1
-- The C compiler identification is GNU 6.3.1
-- The CXX compiler identification is GNU 6.3.1
-- Check for working C compiler: /opt/gcc-aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
-- Check for working C compiler: /opt/gcc-aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/gcc-aarch64-linux-gnu/bin/aarch64-linux-gnu-g++
-- Check for working CXX compiler: /opt/gcc-aarch64-linux-gnu/bin/aarch64-linux-gnu-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_SYSTEM_PROCESSOR value `armv8-a` is unknown
-- Please add this value near 源码位置/mpp/CMakeLists.txt:102
-- Performing Test GCC_HAS_NO_NARROWING
-- Performing Test GCC_HAS_NO_NARROWING - Success
-- Performing Test GCC_HAS_STACK_REALIGN
-- Performing Test GCC_HAS_STACK_REALIGN - Failed
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.28")
-- Checking for one of the modules 'pthread'
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- compile with drm support
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:


    RKPLATFORM




-- Build files have been written to: 源码位置/mpp/build/linux/aarch64

• 双击使用cmake-gui打开/build/linux/aarch64/CMakeCache.txt
  
• 修改CMAKE_INSTALL_PREFIX为想要安装bin include lib的位置，本文使用/home/xzm/3-CSDN_Test_Project/1-mpp/install
  
  
• 点击Generate、Configure
  
• 执行make
  

./make
Scanning dependencies of target osal
[  0%] Building CXX object osal/CMakeFiles/osal.dir/mpp_platform.cpp.o
[  0%] Building CXX object osal/CMakeFiles/osal.dir/mpp_runtime.cpp.o
[  1%] Building CXX object osal/CMakeFiles/osal.dir/mpp_allocator.cpp.o
[  1%] Building CXX object osal/CMakeFiles/osal.dir/mpp_thread.cpp.o
[  1%] Building CXX object osal/CMakeFiles/osal.dir/mpp_common.cpp.o
[  2%] Building CXX object osal/CMakeFiles/osal.dir/mpp_queue.cpp.o
[  2%] Building CXX object osal/CMakeFiles/osal.dir/mpp_time.cpp.o
[  2%] Building CXX object osal/CMakeFiles/osal.dir/mpp_list.cpp.o
[  3%] Building CXX object osal/CMakeFiles/osal.dir/mpp_mem.cpp.o
[  3%] Building CXX object osal/CMakeFiles/osal.dir/mpp_env.cpp.o
[  3%] Building CXX object osal/CMakeFiles/osal.dir/mpp_log.cpp.o
[  4%] Building C object osal/CMakeFiles/osal.dir/android/os_allocator.c.o
[  4%] Building C object osal/CMakeFiles/osal.dir/android/os_mem.c.o
[  4%] Building C object osal/CMakeFiles/osal.dir/android/os_env.c.o
...
Scanning dependencies of target mpi_enc_multi_test
[ 99%] Building C object test/CMakeFiles/mpi_enc_multi_test.dir/mpi_enc_multi_test.c.o
[100%] Building C object test/CMakeFiles/mpi_enc_multi_test.dir/mpp_event_trigger.c.o
[100%] Building C object test/CMakeFiles/mpi_enc_multi_test.dir/mpp_parse_cfg.c.o
[100%] Linking CXX executable mpi_enc_multi_test
[100%] Built target mpi_enc_multi_test

• 执行make install
  

make install
[  9%] Built target osal
[ 10%] Built target mpp_device
[ 14%] Built target mpp_base
[ 14%] Built target vproc_iep
[ 15%] Built target mpp_rc
[ 16%] Built target codec_dummy_dec
[ 16%] Built target codec_avsd
...
[ 96%] Built target mpi_enc_test
[ 97%] Built target mpp_info_test
[ 98%] Built target mpi_test
[ 99%] Built target mpi_rc_test
[100%] Built target mpi_enc_multi_test
Install the project...
-- Install configuration: "Release"

• 查看安装文件夹/install
  

tree
.
├── bin
│   ├── mpi_dec_mt_test
│   ├── mpi_dec_multi_test
│   ├── mpi_dec_test
│   ├── mpi_enc_multi_test
│   ├── mpi_enc_test
│   ├── mpi_rc2_test
│   ├── mpi_rc_test
│   ├── mpi_test
│   ├── mpp_info_test
│   └── vpu_api_test
├── include
│   └── rockchip
│       ├── mpp_buffer.h
│       ├── mpp_err.h
│       ├── mpp_frame.h
│       ├── mpp_meta.h
│       ├── mpp_packet.h
│       ├── mpp_task.h
│       ├── rk_mpi_cmd.h
│       ├── rk_mpi.h
│       ├── rk_type.h
│       ├── rk_venc_cmd.h
│       ├── vpu_api.h
│       └── vpu.h
└── lib
    ├── librockchip_mpp.so -> librockchip_mpp.so.1
    ├── librockchip_mpp.so.0
    ├── librockchip_mpp.so.1 -> librockchip_mpp.so.0
    ├── librockchip_mpp_static.a
    ├── librockchip_vpu.so -> librockchip_vpu.so.1
    ├── librockchip_vpu.so.0
    ├── librockchip_vpu.so.1 -> librockchip_vpu.so.0
    ├── librockchip_vpu_static.a
    └── pkgconfig
        ├── rockchip_mpp.pc
        └── rockchip_vpu.pc


5 directories, 32 files

• 安装MPP结束
  
  

编译live555

• 进入源码目录，新增config.rk3399（根据个人编译器指定内容），内容如下：
  

CROSS_COMPILE?= aarch64-linux-gnu-
COMPILE_OPTS = $(INCLUDES) -I. -O2 -DSOCKLEN_T=socklen_t -DNO_SSTREAM=1 -D_LARGEFILE_SOURCE=1 -D_FILE_OFFSET_BITS=64
C = c
C_COMPILER = $(CROSS_COMPILE)gcc
C_FLAGS = $(COMPILE_OPTS)
CPP = cpp
CPLUSPLUS_COMPILER = $(CROSS_COMPILE)g++
CPLUSPLUS_FLAGS = $(COMPILE_OPTS) -Wall -DBSD=1
OBJ = o
LINK = $(CROSS_COMPILE)g++ -o
LINK_OPTS =
CONSOLE_LINK_OPTS = $(LINK_OPTS)
LIBRARY_LINK = $(CROSS_COMPILE)ar cr
LIBRARY_LINK_OPTS = $(LINK_OPTS)
LIB_SUFFIX = a
LIBS_FOR_CONSOLE_APPLICATION =
LIBS_FOR_GUI_APPLICATION =
EXE =
#个人安装目录，自己修改
PREFIX = /home/xzm/3-CSDN_Test_Project/2-live555/install

• 运行genMakefiles，根据后缀名匹配生成的config文件。
  

./genMakefiles rk3399

• 此时在当前值文件夹下已经生成生成文件，我们需要修改一些进入相应的子文件夹前缀，系统统统改为自己需要安装的地址。
  

#/home/xzm/3-CSDN_Test_Project/2-live555/live/BasicUsageEnvironment/Makefile
#其他几个文件夹也类似，修改之后进行make

• 制作
  

make
cd liveMedia ; make
make[1]: Entering directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/liveMedia'
...
make[1]: Leaving directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/proxyServer'
For more information about this source code (including your obligations under the LGPL), please see our FAQ at http://live555.com/liveMedia/faq.html




make install


make[1]: Entering directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/liveMedia'
install -d /home/xzm/3-CSDN_Test_Project/2-live555/install/include/liveMedia /home/xzm/3-CSDN_Test_Project/2-live555/install/lib
install -m 644 include/*.hh /home/xzm/3-CSDN_Test_Project/2-live555/install/include/liveMedia
install -m 644 libliveMedia.a /home/xzm/3-CSDN_Test_Project/2-live555/install/lib
...
cd proxyServer ; make install
make[1]: Entering directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/proxyServer'
install -d /home/xzm/3-CSDN_Test_Project/2-live555/install/bin
install -m 755 live555ProxyServer /home/xzm/3-CSDN_Test_Project/2-live555/install/bin
make[1]: Leaving directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/proxyServer'

• live555编译完成
  

tree
.
├── bin
├── include
│   ├── BasicUsageEnvironment
│   ├── groupsock
│   ├── liveMedia
│   └── UsageEnvironment
└── lib
    ├── libBasicUsageEnvironment.a
    ├── libgroupsock.a
    ├── libliveMedia.a
    └── libUsageEnvironment.a
7 directories, 213 files


部分demo源码记录
MPP 解码
具体详细的源码可以查看MPP官方文档的mpi_dec_test.c、mpi_dec_mt_test.c、mpi_dec_multi_test.c。


MPP_RET ret = mpi->decode_put_packet(ctx, packet); //put 很多次得到一个frame 输入h264的packet数据
MPP_RET ret = mpi->decode_get_frame(ctx, &frame); //得到解码的yuv数据


MPP 编码
具体详细的源码可以查看MPP官方文档的mpi_enc_test.c、mpi_enc_multi_test.c。


ret = mpi->encode_put_frame(ctx, frame); //阻塞函数，直到输入图像使用完成 输入yuv图像数据
ret = mpi->encode_get_packet(ctx, &packet_g);//得到编码的h264格式数据
// write packet to file here
void *ptr   = mpp_packet_get_pos(packet_g); //得到数据指针位置
size_t len  = mpp_packet_get_length(packet_g);//得到数据长度


p->pkt_eos = mpp_packet_get_eos(packet_g);
//copy数据并写至管道，供live555服务端使用
void *cpy = malloc(len);
memcpy(cpy, ptr, len);
if(mpp_packet_write_to_fifo(cpy, len) < 0)//阻塞
{
     // goto RET;
     printf(" mpp_packet_write_to_fifo err n");
}


live555利用管道实现h264实时视频RTSP推流
上面得到了h264编码后的文件后，写至管道，利用函数写入管道。


int mpp_packet_write_to_fifo(void *ptr, size_t len)
{  
    int ret = 0;
    if(!isOpen)
    {   
        printf("start open the fifo filen");
        pipe_fd = open(fifo_name, O_WRONLY);    //阻塞至读端打开
        if(pipe_fd != -1)
        {
            printf("open fifo successn");
            printf("thread mpp_packet_write_to_fifo, pipe_fd = %dn", pipe_fd);
            isOpen = true;
            //return pipe_fd;
        }
        else
        {
            printf("pipe file open error %sn", strerror(errno));
            return -1;
            // pthread_exit(NULL);
        }
    }
    ret = write(pipe_fd, ptr, len);
    if(ret != len)
    {
        printf("=======Write fifo Err======n");
        return -1;
    }
    return 0;
}
//live555 接口
    OutPacketBuffer::maxSize = 500000;//防止单帧数据过大
   
    char const* streamName = "stream";
    //char const* inputFileName = "my_fifo";
    char const* inputFileName = fifo_name;
    ServerMediaSession* sms = ServerMediaSession::createNew(*env, streamName, streamName, descriptionString);
    sms->addSubsession(H264VideoFileServerMediaSubsession::createNew(*env, inputFileName, reuseFirstSource));
    rtspServer->addServerMediaSession(sms);

live555源码分析

1.针对官方测试程序live/testProgs/testOnDemandRTSPServer.cpp进行部分分析。

• live/testProgs/testOnDemandRTSPServer.cpp main函数中创建服务端，并根据提供的文件名创建相应的类。
  

  // A H.264 video elementary stream:
  //创建h264RTSP服务端，播放的视频是out.h264
  {
    char const* streamName = "h264ESVideoTest";
    char const* inputFileName = "out.h264";//xzm0114
    ServerMediaSession* sms
      = ServerMediaSession::createNew(*env, streamName, streamName,
      descriptionString);
    sms->addSubsession(H264VideoFileServerMediaSubsession
       ::createNew(*env, inputFileName, reuseFirstSource));//根据文件名创建新的H264VideoFileServerMediaSubsession
    rtspServer->addServerMediaSession(sms);


    announceStream(rtspServer, sms, streamName, inputFileName);
  }

• 进一步查看 live/liveMedia/H264VideoFileServerMediaSubsession.cpp
  

H264VideoFileServerMediaSubsession*
H264VideoFileServerMediaSubsession::createNew(UsageEnvironment& env,
      char const* fileName,
      Boolean reuseFirstSource) {
  return new H264VideoFileServerMediaSubsession(env, fileName, reuseFirstSource);
}


H264VideoFileServerMediaSubsession::H264VideoFileServerMediaSubsession(UsageEnvironment& env,
       char const* fileName, Boolean reuseFirstSource)
  : FileServerMediaSubsession(env, fileName, reuseFirstSource),
    fAuxSDPLine(NULL), fDoneFlag(0), fDummyRTPSink(NULL) {
}

• 下一步live/liveMedia/FileServerMediaSubsession.cpp
  
  

FileServerMediaSubsession
::FileServerMediaSubsession(UsageEnvironment& env, char const* fileName,
    Boolean reuseFirstSource)
  : OnDemandServerMediaSubsession(env, reuseFirstSource),
    fFileSize(0) {
    //头文件中定义 char const* fFileName; 子类可以通过继承得到文件名
  fFileName = strDup(fileName);
}


FileServerMediaSubsession::~FileServerMediaSubsession() {
  delete[] (char*)fFileName;
}

• 回到live/liveMedia/H264VideoFileServerMediaSubsession.cpp中
  

//createNewStreamSource 为虚函数 父类中进行使用
FramedSource* H264VideoFileServerMediaSubsession::createNewStreamSource(unsigned /*clientSessionId*/, unsigned& estBitrate) {
  estBitrate = 500; // kbps, estimate


  // Create the video source:
  //xzm0114 通过继承得到的fFilename 创建ByteStreamFileSource
  ByteStreamFileSource* fileSource = ByteStreamFileSource::createNew(envir(), fFileName);
  if (fileSource == NULL) return NULL;
  fFileSize = fileSource->fileSize();


  // Create a framer for the Video Elementary Stream:
  return H264VideoStreamFramer::createNew(envir(), fileSource);
}

• 这里重点看一下继承关系：
  
• 类 H264VideoFileServerMediaSubsession：公共 FileServerMediaSubsession
  
• 类 FileServerMediaSubsession：公共 OnDemandServerMediaSubsession
  
• 打开live/liveMedia/OnDemandServerMediaSubsession.cpp查看调用createNewStreamSource的地方，一共有两处。
  

OnDemandServerMediaSubsession::sdpLines() {
  if (fSDPLines == NULL) {
    // We need to construct a set of SDP lines that describe this
    // subsession (as a unicast stream).  To do so, we first create
    // dummy (unused) source and "RTPSink" objects,
    // whose parameters we use for the SDP lines:
    unsigned estBitrate;
    //调用子类实现的createNewStreamSource
    /* 头文件中定义虚函数
    protected: // new virtual functions, defined by all subclasses
   virtual FramedSource* createNewStreamSource(unsigned clientSessionId,
      unsigned& estBitrate) = 0;
*/
    FramedSource* inputSource = createNewStreamSource(0, estBitrate);
    printf("=====xzm %s:%d:%s=======nn",__FILE__, __LINE__, __func__); //xzm0114
    if (inputSource == NULL) return NULL; // file not found


    struct in_addr dummyAddr;
    dummyAddr.s_addr = 0;
    Groupsock* dummyGroupsock = createGroupsock(dummyAddr, 0);
    unsigned char rtpPayloadType = 96 + trackNumber()-1; // if dynamic
    RTPSink* dummyRTPSink = createNewRTPSink(dummyGroupsock, rtpPayloadType, inputSource);
    if (dummyRTPSink != NULL && dummyRTPSink->estimatedBitrate() > 0) estBitrate = dummyRTPSink->estimatedBitrate();


    setSDPLinesFromRTPSink(dummyRTPSink, inputSource, estBitrate);
    Medium::close(dummyRTPSink);
    delete dummyGroupsock;
    closeStreamSource(inputSource);
    printf("=====xzm %s:%d:%s=======nn",__FILE__, __LINE__, __func__); //xzm0114


  }


  return fSDPLines;
}


//void OnDemandServerMediaSubsession::getStreamParameters函数中
FramedSource* mediaSource
      = createNewStreamSource(clientSessionId, streamBitrate);
    // Create 'groupsock' and 'sink' objects for the destination,
    // using previously unused server port numbers:

• 重点讲一下文件打开和关闭的接口函数
  
• live/liveMedia/InputFile.cpp中 实现了OpenInputFile和CloseInputFile，并在live/liveMedia/ByteStreamFileSource.cpp中调用
  

FILE* OpenInputFile(UsageEnvironment& env, char const* fileName) {
  FILE* fid;


  // Check for a special case file name: "stdin"
  if (strcmp(fileName, "stdin") == 0) {
    fid = stdin;
#if (defined(__WIN32__) || defined(_WIN32)) && !defined(_WIN32_WCE)
    _setmode(_fileno(stdin), _O_BINARY); // convert to binary mode
#endif
  } else {
    fid = fopen(fileName, "rb");
    if (fid == NULL) {
      env.setResultMsg("unable to open file "",fileName, """);
    }
    printf("nnnnn======xzm Open file:%s========nnnnnn", fileName); //xzm0113


  }


  return fid;
}


void CloseInputFile(FILE* fid) {
  // Don't close 'stdin', in case we want to use it again later.
  if (fid != NULL && fid != stdin) fclose(fid);
  printf("nnnnn======xzm Close file ========nnnnnn");//xzm0113
}

• 打开live/liveMedia/ByteStreamFileSource.cpp，函数根据不同的文件类型使用fread或者read，对于管道而言，函数会使用read进行读取。
  

void ByteStreamFileSource::doReadFromFile() {
  // Try to read as many bytes as will fit in the buffer provided (or "fPreferredFrameSize" if less)
  if (fLimitNumBytesToStream && fNumBytesToStream < (u_int64_t)fMaxSize) {
    fMaxSize = (unsigned)fNumBytesToStream;
  }
  if (fPreferredFrameSize > 0 && fPreferredFrameSize < fMaxSize) {
    fMaxSize = fPreferredFrameSize;
  }
#ifdef READ_FROM_FILES_SYNCHRONOUSLY
  fFrameSize = fread(fTo, 1, fMaxSize, fFid);
#else
  //xzm0113


  if (fFidIsSeekable) {
    printf("======Use fread=======nn");//xzm0113
    fFrameSize = fread(fTo, 1, fMaxSize, fFid);
  } else {
    // For non-seekable files (e.g., pipes), call "read()" rather than "fread()", to ensure that the read doesn't block:
   


    fFrameSize = read(fileno(fFid), fTo, fMaxSize);
    printf("======Use read,fFrameSize:%d ,fMaxSize:%d =======nn", fFrameSize, fMaxSize);//xzm0113
  }
#endif
  if (fFrameSize == 0) {
    handleClosure();
    return;
  }
  fNumBytesToStream -= fFrameSize;


  // Set the 'presentation time':
  if (fPlayTimePerFrame > 0 && fPreferredFrameSize > 0) {
    if (fPresentationTime.tv_sec == 0 && fPresentationTime.tv_usec == 0) {
      // This is the first frame, so use the current time:
      gettimeofday(&fPresentationTime, NULL);
    } else {
      // Increment by the play time of the previous data:
      unsigned uSeconds = fPresentationTime.tv_usec + fLastPlayTime;
      fPresentationTime.tv_sec += uSeconds/1000000;
      fPresentationTime.tv_usec = uSeconds%1000000;
    }


    // Remember the play time of this data:
    fLastPlayTime = (fPlayTimePerFrame*fFrameSize)/fPreferredFrameSize;
    fDurationInMicroseconds = fLastPlayTime;
  } else {
    // We don't know a specific play time duration for this data,
    // so just record the current time as being the 'presentation time':
    gettimeofday(&fPresentationTime, NULL);
  }


  // Inform the reader that he has data:
#ifdef READ_FROM_FILES_SYNCHRONOUSLY
  // To avoid possible infinite recursion, we need to return to the event loop to do this:
  nextTask() = envir().taskScheduler().scheduleDelayedTask(0,
(TaskFunc*)FramedSource::afterGetting, this);
#else
  // Because the file read was done from the event loop, we can call the
  // 'after getting' function directly, without risk of infinite recursion:
  FramedSource::afterGetting(this);
#endif
}

• 注：源头修改了一些定义的值，是为了避免单帧过大导致无法读取，具体有
  

// live/liveMedia/StreamParser.cpp
// #define BANK_SIZE 150000
#define BANK_SIZE 3000000 //xzm0113
OutPacketBuffer::maxSize = 500000;//1229xzm live555初始化时执行