ffmpeg rtp and rtcp audio video problem

if I am sending audio and vide thru ffmpeg, I got flickering in the video;
the reason is ffmpeg video 4999 audio 5000
But I am not getting proper audio at client side; I got some glitches .
Reason :
---------------
1.we have to choose the even number of rtp port;rtcp packets will be sent by the ffmpeg ( rtp port + 1) = rtcp port number;
2. rtp video is sent in 4999 and ffmpeg will send the rtcp info at 5000
3.At the same time ffmpeg will sends audio at RTP port 5000 and RTCP at 5001;
4.So FFMPEG is sending the RTP and RTCP packets in a same Port that is 5000;

Source Filter's Output Pin Doesnt match any known format type

Source Filter's Output Pin Doesnt match any known format type like Format_VideoInfo,Video2...
---------------------------------------------------------------------------------------------
I developed the Dshow Player to render the Ball filter;
CoCreateInstance() fn is working fine ;
I get the Output Pin of the ball filter and render it;
Rendering is failed;I checked the Ball Filter's Output Pin with Different Video types like VideoInfo, VideoInfo2,MPEG1Video ,MPEG2Video and DVVideo and none of them is matching to the media type of the source filter;
Solution:
-----------
Reason is silly. I got this error due to I am not adding the Ball Source Filter's IBaseFilter object to the FilterGraph;
So I just added the IBaseFilter Object to the FilterGraph leads to working graph;

Problems Faced in registering the Filter DLL in Windows CE

Problems Faced in registering the Filter DLL in Windows CE:
-----------------------------------------------------------
I faced the problem in registering the DLL ;
Solution:
-----------
There is no such file to register the DLL in Windows Mobile Emulator evnvironment like regsvr32 ;
So I developed the my own version of RegSvr32 application with the help of MSDN KB Code;
My Register application will do the following:
1.Loads the DLL
2.GetProcAddress() fn to get the address of the "DllRegisterserver" and call it;
3.Unloads the DLL;
Now it was registered in to the registry;








Because the .def file is not included in DLL project input; what it means was
within this .def file we specified the DLL functions as follows:
DllRegisterServer
DllUnRegisterServer
Then only we will get the correct address of the DllRegisterServer fn. otherwise we will get Null pointer;



One More problem I faced while calling the DllRegisterServer() fn address from My Register application:
The application was hanged and not giving any error, I got the execption within that fn;
I am not able to figure out the problem;


Solution:
-------------
So I checked the MSDN documentation as follows:
For Windows CE,we need to do the following:
STDAPI DllRegisterServer()
{
return AMovieDLLRegisterServer( );// Windows Desktop- AMovieDLLRegisterServer2(TRUE) - these are not supported in WinCE
}
STDAPI DllUnregisterServer()
{
return AMovieDLLUnRegisterServer( ); //Windows desktop-AMovieDLLRegisterServer2(FALSE)to unregister-these are not supported in WinCE
}
However, within DllRegisterServer and DllUnregisterServer you can customize the registration process as needed.
One more thing we need to
CFactoryTemplate g_Templates[] =
{
{
g_wszName, // Name.
&CLSID_SomeFilter, // CLSID.
CSomeFilter::CreateInstance, // Creation function.
NULL,
&sudFilterReg // Pointer to filter information.
}
};
int g_cTemplates = sizeof(g_Templates) / sizeof(g_Templates[0]);

LPAMOVIESETUP_FILTER CSomeFilter::GetSetupData() //we must implement this one for Windows CE
{
return (LPAMOVIESETUP_FILTER) & sudFilterReg;
}

Windows CE Directshiw Filter Error

Windows CE Directshow Filter Error :
-------------------------------------
I developed the Ball Source filter in Windows CE;
I got these errors:
Creating library Pocket PC 2003 (ARMV4)\Release/Ball_Filter.lib and object Pocket PC 2003 (ARMV4)\Release/Ball_Filter.exp
1>fball.obj : error LNK2019: unresolved external symbol "public: __cdecl CSourceStream::CSourceStream(wchar_t *,long *,class CSource *,wchar_t const *)" (??0CSourceStream@@QAA@PA_WPAJPAVCSource@@PB_W@Z) referenced in function "public: __cdecl CBallStream::CBallStream(long *,class CBouncingBall *,wchar_t const *)" (??0CBallStream@@QAA@PAJPAVCBouncingBall@@PB_W@Z)
1>fball.obj : error LNK2001: unresolved external symbol "protected: virtual long __cdecl CSourceStream::QueryId(wchar_t * *)" (?QueryId@CSourceStream@@MAAJPAPA_W@Z)
1>fball.obj : error LNK2019: unresolved external symbol "public: __cdecl CSource::CSource(wchar_t *,struct IUnknown *,struct _GUID)" (??0CSource@@QAA@PA_WPAUIUnknown@@U_GUID@@@Z) referenced in function "private: __cdecl CBouncingBall::CBouncingBall(struct IUnknown *,long *)" (??0CBouncingBall@@AAA@PAUIUnknown@@PAJ@Z)
1>fball.obj : error LNK2001: unresolved external symbol "public: virtual long __cdecl CSource::FindPin(wchar_t const *,struct IPin * *)" (?FindPin@CSource@@UAAJPB_WPAPAUIPin@@@Z)
1>fball.obj : error LNK2001: unresolved external symbol "public: virtual long __cdecl CBaseFilter::JoinFilterGraph(struct IFilterGraph *,wchar_t const *)" (?JoinFilterGraph@CBaseFilter@@UAAJPAUIFilterGraph@@PB_W@Z)
1>fball.obj : error LNK2001: unresolved external symbol "public: virtual long __cdecl CBaseFilter::QueryVendorInfo(wchar_t * *)" (?QueryVendorInfo@CBaseFilter@@UAAJPAPA_W@Z)
1>Pocket PC 2003 (ARMV4)\Release/Ball_Filter.dll : fatal error LNK1120: 6 unresolved externals


Solution Steps:
--------------------
I modified the things in "Release" Mode:

it may be due to calling convention mechanism;
But ARM doesnt have calling convention compiler option( /Gr, Gd,Gz);
Calling convention is applicable only to x86 machines now I am using ARM;

I doubt whether the Strmbase.lib is built in x86; if it is so means, Previously I developed the dshow Player application
with this same "strmbase.lib"; So there is nothing wrong in lib;
I closely watched the all the error fns; they are all contains wchar_t;
It may due to some unicode problem?
Solution:
----------
Project->Properties->C++ ->Language ->
Treat wchar_t as Built In Type:Yes

I just modified this property to No;
Treat wchar_t as Built In Type: No

Now all the linker errors are resolved;
Now I am not having any error;


Note:
---------
Even though I did all the above things in Debug Mode, I got the following error:
Creating library Pocket PC 2003 (ARMV4)\Debug/Ball_Filter.lib and object Pocket PC 2003 (ARMV4)\Debug/Ball_Filter.exp
fball.obj : error LNK2001: unresolved external symbol "public: virtual unsigned long __cdecl CBaseFilter::NonDelegatingRelease(void)" (?NonDelegatingRelease@CBaseFilter@@UAAKXZ)
Pocket PC 2003 (ARMV4)\Debug/Ball_Filter.dll : fatal error LNK1120: 1 unresolved externals
For Debug Mode:
I have added this code within the CSource derived class;
public:
STDMETHODIMP_(ULONG) NonDelegatingAddRef()
{
return InterlockedIncrement(&m_cRef);
}


//-----------------------------------------------------------------------------
// Decrement the reference count for an interface
// ////////////////////////////////////////
STDMETHODIMP_(ULONG) NonDelegatingRelease()
{
if(InterlockedDecrement(&m_cRef) == 0)
{
delete this;
return 0;
}
return m_cRef;
}
Now it is working fine;

I have successfully build the filter code;

Audio and Video Port Problem in Audio Testing

RTP AMRTesting :
=================
1.I combined the multiple RTP packets in to a Single packet and passed it to the AMR Paser application;
2.Next added the AMR file extension and AMR start codes and try to play it with QT Player;
Created AMR file is wroking with QT Player; It is working.

3.Pls kindly check the Market Bit of the RTP header format;

if we are sending the audio and video thru INVC transcoder,I printed the RTP packet Number as follows:
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 60, Buffer size = 1389, Marker = 0
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 61, Buffer size = 1389, Marker = 0
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 62, Buffer size = 1389, Marker = 0
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 63, Buffer size = 1389, Marker = 0

ffmpeg sends amr audio to RTP.I got the following:
RTP packet Number = 3, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 3
RTP packet Number = 4, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 4
RTP packet Number = 5, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 5
RTP packet Number = 6, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 6
RTP packet Number = 7, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 7
RTP packet Number = 8, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 8
RTP packet Number = 9, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 9


while sending audio and video we have the problem;One more thing if we send audio and video, QuickTime player is able to render the audio.


So without the packet number we need to do parsing;
Every Audio Frame Beginning is marked by the Marker bit as Zero;otherwise it will be 1 or TRUE;
If I developed the amr parser for this scenario, It causes the Screech Sound in audio with particular interval.At client side, I received the RTP packets and Dumped it into a file also givesthe same problem;

Solution:
-----------------

Finally I got the Solution:
we are sending video data at 5000 Port We are sending audio data at 5001 Port ;So it causes the problem;
Instead of it If I changed the things as
video as 4500 and Audio as 5000 means there will be no problem So Every RTP amr packet is having header Info;
For Audio and video There must be some difference.So that we can avoid this situation;

AMR over RTP:

RTP Packet Size 1389

RTP header size 12

--------

1377

-----------

(1377 / Frame Size) - 1 = Number of ToC Entries; each frame has ToC entry in RTP

Or just count the number of bytes (ToC entries) to identify the frames available in a RTP packet;

How can we extract the AMR mode information from the RTP packet?

First check the CMR (Codec Mode Request) .

It is having bandwidth. For Each bandwidth, the frame size is fixed;

CMR Mode Frame size (bytes)

0 AMR 4.75 13

1 AMR 5.15 14

2 AMR 5.9 16

3 AMR 6.7 18

4 AMR 7.4 20

5 AMR 7.95 21

6 AMR 10.2 27

7 AMR 12.2 32

Every AMR frame is having 20 ms of audio data;

So if 40 frames are available means

40 * 20 = 800 milli seconds (play time of that frames);

1000 milliseconds = 1 second

From Source Filter, Based on Number of Frames we need to set the Start and Stop timestamps.

AMR over RTP is as follows:

+----------------+-------------------+----------------

| payload header | table of contents | speech data ...

+----------------+-------------------+----------------

Payload header is 4 bits;

First 4 bits are CMR (Codec Mode Request); From Codec Mode Request value, we can identify the frame size;

ToC (Table of contents) -

Each and every frame has an entry for the ToC;

If RTP packet is having 43 audio frames means that much of Toc Bytes must be available.

After F0, Remove the same types of bytes; (this byte indicates the audio bit rate information)

But How can we know the Frame Type?

1011 1100 (BC) 12.2 kbps

1011 0100 (B4) 10.2 kbps

1010 1100 (AC) 7.95 kbps

RTP packet:

---------------

If a terminal has no preference in which mode to receive, it SHOULD set CMR=15 in all its outbound payloads

Each RTP AMR data begins with F0 and then ToC entries like 0xac, 0xbc, 0xb4 as repetitive.

If the RTP packet has N frames, RTP packet is having N number of TOC Entries.

From the TOC Entry we can define the frame size of the audio frame;

TOC Entry will be in the following form:

---------------------------------------------

F (1 bit) | FT (4 bits) | 1 (1 bit)

---------------------------------------------

1 0111 1 00 -12.2 kbps ( 0x BC)

1 0110 1 00 -10.2 kbps ( 0x B4)

1 0101 1 00 - 7.95 kbps ( 0x AC)

------------------------------------------------

FO BC BC BC BC BC BC BC BC BC 3C

After the TOC contents, the First start code acts as a frame header;

3C is the frame header available in an audio frame and every audio frame must begins with 3C;

From the bit rate, we can determine the Frame Size;

CMR Mode Frame size (bytes)

0 AMR 4.75 13

1 AMR 5.15 14

2 AMR 5.9 16

3 AMR 6.7 18

4 AMR 7.4 20

5 AMR 7.95 21

6 AMR 10.2 27

7 AMR 12.2 32

Frame Size is including a frame header;

But RTP packet is having the frames as follows:

First Frame alone has the 1 byte audio frame header; rest of the frames will not have header; we need to add it manually;

Ac 12 20 39 40 29 20 39 33

Ac is a frame header and from the header onwards we can identify the number of bytes per frame; assume that if the frame header info as 4.75kbps having frame size 13 means from the frame header, count the 13 bytes; then next insert the First frame’s frame header and count the 13 bytes from the header and then insert the frame header for 3^rd frame;

1^st Frame Header bytes After 13 bytes insert the First Frame’s header, then next insert the frame header after the 26 bytes and next insert the frame header after the 39 bytes. Do it repeatedly.

If we have not inserted the frame header at the every frame start, then it will be decoded by the AMR decoder but u will not have any hearable audio.

Two types of AMR:

-----------------------

1.AMR or AMR_NB ( AMR narrow band) - Mono channels ( 8 bit audio)

2.AMR_WB ( AMR wide band) - stereo channels ( 16 bit audio)

we are using AMR_NB ;

Every AMR file begins with following Hex start codes:

23 21 41 4d 52 0a ( Equal string is: #!AMR.)

if we open the amr file, without these start codes means it will display like

---------------------------

Error

---------------------------

Error -2048: Couldn't open the file Quotes.amr because it is not a file that QuickTime understands.

---------------------------

OK

---------------------------

Every audio frame is having one byte frame header;

format is as follows:

---------------------------------------------

F ( 1 bit) | FT( 4 bits) | 1 ( 1bit)

---------------------------------------------

Frame Type	Mode	Frame Size(Including 1 byte header)
0	AMR 4.75	13
1	AMR 5.15	14
2	AMR 5.9	16
3	AMR 6.7	18
4	AMR 7.4	20
5	AMR 7.95	21
6	AMR 10.2	27
7	AMR 12.2	32

if we have a file with 12.2 kbps means, we can check this by the frame header;

and moreover we can note that every frame have 1 byte header;But it is same as start. Assume that first frame's header is 3C then every frame must begins with header start code ;

if the header is having proper start code and the next it will check whether every frame is having same header info; otherwise it will reports the error as follows:

"Quicktime media file is having incorrect duration"

I have to do one more check;

if every frame is having different header start code whether the QuickTime play that AMR file ?...

3C is a frame header for first frame;

38 is a frame header for second frame;

Both frame headers indicate the same Frame Type;

whether it will works ?

Result :

----------

it works fine;

Test case 2:

-------------------

Instead of 0x38, I inserted the 0x20.

Then I didnt get any sound while playing it in QuickTime player and moreover the Quicktime is not giving any errors;

Instead of 0x20, I made it as 0x20, then I faced the same problem.

Every AMR frame is having 20 ms of data;

So if 40 frames are available means

40 * 20 = 800 milli seconds( play time of that frames);

1000 milliseconds = 1 second

How to stream AMR to RTP and Play it in QuickTime

Use AMR file as source to stream AMR to RTP. ffmpeg -re -i "d:\media\vel1.amr" -acodec libamr_nb -ar 8000 -ab 12.2kb -ac 1 -f rtp rtp://192.168.1.198:5000/

it plays the audio in Quicktime.
AMR.SDP File contents:
-----------------------
v=0
o=Mass 3123312 121232 IN IP4 192.168.1.198
s=Rtsp Session
c=IN IP4 192.168.1.198
t=0 0
a=range:npt=0-
m=audio 5000 RTP/AVP 97
a=rtpmap:97 AMR/8000
a=fmtp:97 octet-align=1