Use the following steps to open the test kit.
- In Visual Studio, open a Windows Phone application solution that targets Windows Phone OS 7.1 or later.
- In Solution Explorer, select the project you want to test.
- On the Project menu, choose the Open Marketplace Test Kit option.
What Will the Test Kit Tell Me?
Running the test kit on your application will help you determine whether or not the application will pass Marketplace certification. If a test fails, the test kit provides details about what you need to fix in the application prior to submitting it for Marketplace evaluation. The following lists some of the tests in the test kit.
- Whether the images provided meets certification guidelines
- What capabilities the application uses
- Whether the application screenshots meet the certification guidelines
- Whether the XAP meets size guidelines
- Whether the application starts quickly enough to meet certification guidelines
- If the application does not exceed memory usage requirements
- Proper use of the back button in the application
- Application responsiveness
- Whether the application closes properly
Overview of Tests in the Test Kit
The tests in the test kit are categorized by whether they can be run automatically or need partial or full participation from a developer. Each tab in the test kit contains either the application details or a category of test. Each test category lists a series of tests with a name and description. The test kit contains the follow tabs.
- Application Details
- Automated Tests
- Monitored Tests
- Manual Tests
The following sections discuss these tabs in more detail.
Application Details
The Applications Details tab lists the application package and enables you to specify images that will be evaluated later in the testing procedure (see the screen shot at the start of this article for an example)
The Application package box lists the path to the application package (XAP file) for your application, which by default is the BinRelease folder of the currently selected project. .
The Applications Details tab also enables you to specify the images that will represent your application in the Marketplace. These images will be tested as part of the test kit. If you do not specify the required images, your application will not pass the tests in the test kit. The following table lists the images that you can specify.
Automated Tests
The Automated Tests tab provides tests that evaluate the basic criteria of your application, such as application size, capabilities required by your application, and the presence and size of application images.
NB Note that the project needs to be built in release mode in order to run these tests:
You run the tests by clicking the Run Tests button. When the tests complete, you will receive a pass or fail result for each test. You will also receive a description of the test and any error messages that are relevant. You can use the results of the Capability Validation test to update the WMAppManifest.xml file with the correct information. If a test fails you should use the provided information to fix the issue and run the tests again. You should repeat this process until your application has passed all the tests in this section.
My initial automated test results give detail on changes required prior to marketplace submission:
(Click on above image to enlarge)
Monitored Tests
The Monitored Tests tab provides a suite of tests that analyse the performance and reliability of your application during use on a device.
(Click on above image to enlarge)
NB the above results come from an HD7 running the development release of Mango.
Manual Tests
The Manual Tests tab provides a series of test cases that you should complete with your application running on a Window Phone device:
This essentially provides a checklist of manual tests which you manage yourself via a dropdown box in the first column:
This should help improve on the quality of apps submitted to the marketplace if used correctly. It should also ease the burden on the testers in the marketplace ingestion process!