What kind of testing needs to be performed on handsets?
There are many kinds of testing that needs to be performed on handsets. The most basic is the Conformance test that is specified in 3GPP specification 34.123. Except that most handset manafacturers perform more tests using the Network Simulators. Then they have to perform IOT (Inter Operability test), Live air testing, Field Testing and KPI (Key Performance Index) testing, etc.
There are many different types of Conformance tests that the handset should be able to pass. Generally the handset has to pass 95% of the tests to be GCF certified. The following are the main categories of Conformance:
- RF conformance - This is based on 3GPP specifications 34.121. Recently due to its size, it has been split into 34.121-1 and 34.121-2.
- Protocol conformance - This is based on 3GPP specifications 34.123. It is split into 34.123-1, 34.123-2 and 34.123-3.
- USIM conformance - This is based on 3GPP specifications 31.122.
- Acoustic conformance
Anyway GCF Certification is only done for Protocol Conformance tests based on 34.123. These tests are written in TTCN and are contained in 3GPP TS 34.123-3 drafted by ETSI on behalf of 3GPP. They are debugged and verified by individual SS vendors on their platforms prior to submission for 3GPP TSG-T1 approval. Once approved by 3GPP test cases are validated by an independent test house and submitted for approval by the GCF. The GCF Conformance Test Toolkits provide the approved test cases and validated protocol test systems required by UE manufacturers and test houses for independent validation and GCF certification of a 3G UE.
To learn more about the GCF certification see: http://www.globalcertificationforum.org/static/?page=GCF_overview.htm
No its not. If the handset passes Conformance Test, it can get GCF (Global Certification Forum) certification. That proves that the UMTS handset is compliant to 3GPP specifications but is not sufficient basis for the release of handset. All operators go through their own rigorous testing before the handset can be released. Also there is a requirement by all operators that the handset platform should have undergone atleast two IOT's.
Yes its possible. Finally it all depends on the operator and at present (June 06) most of the UMTS handsets are not GCF certified.
IOT's are very different to conformance testing. Some of the UE's that pass all conformance tests do perform badly in IOT's. Some of the differences are as follows:
- In Conformance tests, the number of neighbour cells are very limited. It is nowhere like a real network. On the other hand in IOT's the number of cells in Sib 11/12 are much more (sometimes upto 32). This means that there are much more SIB segments as compared to conformance. UE's can have problem handling these even though they pass Conformance with no problems.
- In Conformance tests the Cells are switched off and on immediately, whereas in IOT's, their power is changed gradually, this can sometimes expose some flaws in the UE's.
- In Conformance, generally the data path is not checked. For example a Voice call is started and certain scenarios are simulated (like Handover, Relocation, etc) the voice quality is not checked. We have come across UE's that do pass the complete scenarios but instead of the voice call, all you hear is the Ciphering noise. We have also seen some UE's where the voice path gets dropped after 5 minutes in the call. All these cannot be tested in Conformance tests.
- Similarly on the packet side you can check the Ping/Ftp is being properly performed in IOT's that cannot be seen in Conformance tests.
No there are no written specs. All networks do have their own set of tests that they need to perform for IOT's but they are different for each network operator. The most basic set of tests that need to be performed for IOT's are:
- Voice calls
- To Test:
- Call setup success
- Call drops
- Inter System Hand Off (ISHO)
- BLER
- Video Calls
- To Test:
- Call Setup Success
- Call Drops
- BLER
- PS calls
- To Test:
- PDP Context Activation Success
- PDP Context Drop Call Ratio (DCR)
- ISHO
- Throughput DL and UL
- Ping Round Trip Time (RTT)
All Network operators have to perform IOT's to prove that their UMTS network works with different UE's. They are not all the same. Some of them are really tough. We have found that Ericsson and Nortel are the toughest.
The IOT's are still controlled environment and there is no check for RF, etc. In Field testing the UE's are exposed to many networks simultaneously and the UE has to use the air interface that can make lot of difference. Field testing also involves testing for how the UE behaves in mobility environment and how good it is in performing handovers, etc.
We know of a UE that would crash when going through a particular street. After lots of investigations it was found that the SIB segmentation in one particular cell was different and the UE could not handle it properly, hence the crash. There are many many other problems that only get exposed in Field testing.
The first thing you should know is what particular scenario you are interested in. Once you know that then you should get the latest version of 34.123-1 from the 3GPP server and try to find a test case that will match your scenario. Once you have found that then you can find the UE log on the 3GPP server for that particular scenario. Unfortunately you will have to go through all the directories to find the test case. There is no shortcut i am aware of. Some of the old tests are not in TSG-RAN but in TSG-T.