Localized IDE Validation Test Suite
This page is describing how to user automated test suite for your localized NetBeans IDE. You can check basic IDE functionality in the build environment.
Overview
There is an automated test suite to test the NetBeans IDE. The base technology is being developed as part of NetBeans project:
- Jemmy: Java library that is used to create automated Java GUI tests
- Jellytools: Java library to create UI tests for NetBeans IDE. It's based on above Jemmy.
- XTest: Test framework
The most visible implementation is Commit Validation Test Suite which is described as part of commit process. This localized IDE validation test suite is a configuration to run a part of commit validation test suite for the localized builds.
Commit Validation Test Suite and Localized IDE Validation Test Suite
When commit-validation ant target is executed in master build script nbbuild/build.xml, it checks the following things:
- Compare result of a build with golden files verifying various aspects of the exported interfaces
- Unit tests for core, core/winodws, and core/javahelp
- QA functional testing of IDE
Localized IDE validation suite execute only the QA functional testing of IDE.It's used to see there are no regressions in basic IDE functionality. To do this, ant script is provided as translatedfiles/test/build.xml.
Tested items
The following IDE basic features are tested:
- Showing Help top page
- Showing Option dialog
- Standard Java Project Wizard
- New Java class file creation
- Project View, and File view
- Editing and execution, Java class
- Debugging, Java class
- JUnit: test creation and test execution
- XML file creation, DTD generation, SAX scanner creation
- Web Application creation, and one JSP execution
- Showing UpdateCenter wizard
- JDBC data source creation
- Window System (switch MDI and SDI)
- Module installation (enable/disable Image module)
You can see more detail in the test protram
translatedfiles/test/patches/ide/test/qa-functional/src/validation/IDEValidation.java
TODO for localized IDE validation test suite
The current localized IDE validation suite is not integrated into NetBeans commit-validation testing framework as described in Commit Validation Test Suite page. It's only execute the following test program which is part of commit validation test suite:
ide/test/qa-functional/src/validation/IDEValidation.java
TODO for IDEValidation.java
The above IDEValidation.java can not be used for localized build as it is. It needs to be modified to run with localized IDE. The test script overrides original source file with modified one. Anyone who is going to test, does not have to care about modification, but in general, it's better to implement that modification into original IDEValidation.java.
Original file: ide/test/qa-functional/src/validation/IDEValidation.java
Modified file: translatedfiles/test/patches/ide/test/qa-functional/src/validation/IDEValidation.java
Steps
- Build English IDE, and run English suite
- Build localized NetBeans IDE
OR
Download localized NetBeans IDE
- Run localized ide validation suite
1. Setup build environment, and run English suite
Please see how it works in English build environment. The suite for localized build require the English environment. To build NetBeans locally, see the build information at netbeans.org for detail:
Only the steps are listed here:
- Setup your CVSROOT environment value is
:pserver:anoncvs@cvs.netbeans.org:/cvs
anoncvs can be replaced with your account.
- Check out all required source files
- Build English IDE
- Run English commit validation suite
Here is an example steps
$ export CVSROOT=:pserver:anoncvs@cvs.netbeans.org:/cvs
$ cvs login
just hit return key
$ cvs co -P -r release50 localizable_nowww
You can replace
anoncvs with your account. By executing
cvs co, all the required source files are checked out.
If you've already checked out source files for localization, run the following command to check out test suite.
$ cd your_source_directory
$ cvs co -P -r release50 translatedfiles/test
$ export ANT_HOME=/home/foo/apache-ant-1.6.5
$ export JAVA_HOME=/usr/jdk/latest
$ export PATH=$ANT_HOME/bin:$JAVA_HOME/bin:$PATH
$ cd your_source_directory/nbbuild
$ ant build
build ant target builds NetBeans IDE in
nbbuild/netbeans directory, and zip file is created as:
NetBeans-release50-YYMMDD.zip
YYMMDD: time stamp
Now you are ready to execute English commit validation suite.
$ ant commit-validation
The test report is created at:
ide/test/results/index.html
The all testing should finish without any errors, if you see some errors or unexpected exceptions, please make sure you are building stable 5.0 build, or other stable build.
2-a. Build localized NetBeans IDE
If you are going to test downloaded localized NetBeans IDE, skip this section and go to 2-b. Download localized NetBeans IDE.
To build localized IDE, see the following HOWTOs:
Only the steps are listed here:
- Build English zip file
- Build localized IDE
- Run test suite
Here is an example steps
$ cd your_source_directory/translatedfiles
$ ant buildide -Dlocales="ja"
-Dlocales="ja" needs to be changed for locale which you are going to build (e.g. one of the az, cs, de, es, fr, ko, nl, pt_BR, ru, sq, and sv).
You may see an error messages which says "Missing build zipfile in .../nbbuild/NetBeans-release50-YYMMDD.zip" (YYMMDD is time stamp). In that case, please make sure zip file is located in nbbuild directory. And if "YYMMDD" is not today's time stamp, you have to specify build number as the following example:
$ ant buildide -Dlocales="ja" -Dbuildnum="release50-060703"
You have to specify same string in English zip file name. If you have English IDE:
nbbuild/NetBeans-release50-060704.zip
the argument should be
-Dbuildnum="release50-060704"
.
2-b. Download localized NetBeans IDE
You can download and install localized IDE from the following download page:
http://www.netbeans.info/downloads/download.php?a=n&p=1
3-a. Run validation suite for local build
If you are going to test downloaded and installed NetBeans IDE, skip this section and go to 3-b. Run validation suite for downloaded/installed build.
The steps are:
- Extract image into test directory
- Execute test suite
Here is an example steps:
$ cd your_source_directory/translatedfiles/test
$ ant setup-build -Dbuildnum=release50-060620 -Dlocale=ja
By executing
setup-build target, localized IDE is extracted into
test directory. Please make sure to specify
-Dlocale and
-Dbuildnum. If you have localized IDE zip file as:
translatedfiles/NetBeans-release50-060701-ja.zip
the arguments should be:
-Dbuildnum=release50-060701
-Dlocale=ja
Now you are ready to run localized ide validation:
$ ant ide-validation
Test results are stored into the same file as English commit validation suite.
ide/test/results/index.html
3-b. Run validation suite for downloaded/installed build
Validate downloaded/installed IDE
If you've downloaded localized IDE, and installed it, the steps are:
- Execute test suite
Here is an example steps:
$ cd your_source_directory/translatedfiles/test
$ ant ide-validation -Dnetbeans.dest.dir="/home/foo/netbeans-5.0"
/home/foo/netbeans-5.0 in the above example means NetBeans IDE install directory. It needs to be replaced with your actual installed location.
Test results are stored into the same file as English commit validation suite.
ide/test/results/index.html
Evaluate localized IDE validation results
Please refer the Commit Validation Test Suite information to evaluate results. All the cases should be finished without any errors. Please email
dev@translatedfiles.netbeans.org
, if you see some unexpected errors, or exceptions.
NOTE
When I was investigating, an unexpected exception AsserstionFailure was thrown, but it can not be reproduced regularly. Even when you face that exception, you may not see that exception next time. Please email
dev@translatedfiles.netbeans.org
if you see that exception in every testing.
