BPS has deep expertise in the skills and tools required for performance testing, including requirements analysis, test automation, system analysis, and technical reporting. All of our staff are experienced in a wide array of performance testing tools, including all best-of-breed and industry standard tools. Most hold certifications in relevant test tools, test scripting languages, and test methods. Additionally, most of our testers have software development experience, giving them a solid foundation for automated test script development.
Our testing process parallels the system development life cycle, and includes all activities and deliverables needed to ensure a disciplined, responsive approach to performance testing. It is rooted in recognized standards, such as IEEE Std 1012 for Software V&V, and is customized based on the client and project.
Highlights of our process are summarized below:
- Plan. Our planning for each testing engagement includes assigning test engineers, documenting test objectives to determine test type, establishing pass/fail criteria, collecting critical use cases, creating the preliminary test schedule, identifying any service level agreements or non-functional/performance requirements, analyzing requirements to verify their testability, and identifying deployment schedules and relevant system documentation.
- Prepare. We prepare for testing in support of test objectives by developing and integrating automated test scripts, test data, historical and projected usage, and ensuring test script traceability to target critical test cases selected during planning.
- Execute. We execute the automated test scenario by verifying system instrumentation, configuring measurements of interest, verifying test infrastructure, and running tests with appropriate ramp-up/sustain/ramp-down periods.
- Analyze. We analyze raw data using analysis tools to derive key front-end performance measures and correlate them with back-end measurements. We review test logs, identify causes for errors, and verify test results with a short scenario rerun if necessary.
- Report. We prepare test reports by combining analysis and observations into a reporting template based on best practices covering essential measures of speed, throughput, and reliability. We submit the test report covering discrepancies, critical test observations, and recommendations after a mandatory peer-review by team members to ensure deliverable quality.
- Closeout & Improve. We close testing engagements by saving all essential elements including documentation, test scripts, analysis, test reports, test plans and test data into the enterprise CM repository. We update lessons learned and workarounds in a shared folder in addition to sharing them during technical reviews.
Success Stories
USPTO Performance Analysis Testing
BPS performs performance analysis testing of all 160+ systems that support the mission and operations of the USPTO, conducts detailed analyses to identify and help isolate issues, and provides recommendations regarding production deployment. We coordinate closely with many engineering and support teams to help test applications against tight schedule constraints. We have performed testing for over 360 releases of a variety of systems in the last two years alone.
Our current work at USPTO exemplifies our expertise in performance testing: disciplined enterprise-level performance testing, strong Agile/DevOps practices, a high level of automation, a stable and highly skilled team, effective cross-organizational coordination, and high-quality results covering hundreds of systems. We have achieved this success while performing a major shift from Waterfall to Agile/DevOps models. We use a disciplined approach and incorporate industry best practices to provide quality assurance through testing software applications and software components for USPTO. We use a proven test architecture to verify that the systems under test satisfy performance requirements/SLAs through our testing effort, and we work with the Task Order Manager and Test Managers to ensure each engagement is planned, staffed, and managed to ensure its success.
HHS Grants.gov Program Technical Support
Grants.gov provides a shared service accessible to 26 agencies and thousands of public organizations to support pre-award grants management processes, including the ability to find and apply for federal grants worth $350 billion annually.
BPS provides a comprehensive range of testing services for Grants.gov, including performance testing services that cover end-to-end functionality available for grantors, applicants, and administrators. We also conduct performance testing of the system-to-system interface used by all the 26 grant-making federal agencies to integrate with Grants.gov. In addition, BPS’ testing scope includes the next generation transformation system being built using the Agile/DevOps model.