A major goal of Homeland Security
Presidential Direction (HSPD) 12 was to have an interoperable credential
across logical and physical contexts for employees and contractors. The goal remains elusive. The APL was developed to help meet this
goal. Some changes need to take place to
better achieve this.
The APL consists of products and systems. In both cases the APL tests items against a
test script(s). The testing program does
not test product items with each other or in real world configurations. There
is no test against a matrix of devices for interoperability. As the list expands and data models evolve it
will be increasingly difficult to achieve interoperability based on the current
testing done.
In the case of the APL authentication systems test you find
items that are bolted on to make up for deficiencies in the typical items found
in an access control system. If you look
at the list you will see that no typical access controller (door controller or
domain controller) is on the Approved Products List. Making testing configurations map to system
configurations would help achieve interoperability goals.
A representative infrastructure for testing does not exist
(it is also limited in agencies but that is a different matter). As an example there is a Central Certificate
Validator (CCV) yet for authentication system testing applicants do not use the
CCV. Instead each item under test must
provide their own on line certificate status protocol responder. The APL should make a representative
development environment generally available and for the same environment to be
used for test.
The 33 categories need to be revisited. The test categories (items) in some cases do
not represent components in a physical access control system. The first generation APL products that have
come to market are not based on the enterprise architecture requirements but
rather are based on the test item product and system categories. Product categories that are more aligned with
solution components and tests that more aligned with best practices in product
assurance and interoperability would make the APL better.
This is the third review over 3+ years (see blog) of the Federal
Information Processing Standard 201 (FIPS 201) Government Services
Administration (GSA) Approved
Products List (APL) also known as the FIPS 201 Evaluation Program for
Products & Services. The APL process was started 8 years ago by HSPD
12. Directly related to that process and
since IDmachines’ last review FIPS
201-2 Draft Personal Identity Verification (PIV) was issued March 2011 and
the comment period closed. Changes in a
number of areas are expected (see blog on NIST top 10). Also the Federal
Identity Credentialing and Access Management
(FICAM) 2.0 was issued in December 2011. Indirectly the National Strategy for Trusted Identities in
Cyberspace (NSTIC) was issued in April 2011. Products and systems on the APL are directly
and indirectly impacted by significant other Federal cybersecurity and
information security initiatives and publications (see below), many of which
have come to pass recently.
Card
Reader – Biometric Authencation
|
Card
Reader - Biometric Authentication
|
Card
Reader - CHUID Authentication Reader (Contact)
|
Card
Reader - CHUID Authentication Reader (Contactless)
|
Certificate
Validator (without authentication)
|
PIV
Card Delivery
|
There are six hundred and six products from 111 companies
listed in 27 of 33 categories. In the
first table the categories have no products.
Little changed from 2011 when there were also six categories without
products. The first product has been
listed in the Biometric Authentication System category. The Certificate Validator and Server-based
Certificate Validation Protocol (SCVP) client categories have been modified to
have listing categories of with or without authentication. There is only one supplier in 5 categories.
- 1/3rd of categories have 0 or 1 supplier
- Nearly ½ (48.4%) have 2 or less suppliers
- Over 1/3rd are in the transparent reader category
- About 70% are in the top 5 categories (69.47%)
- Over 85% (86.3%) are in the top 10
The following table shows the counts of products and
companies in the categories over time.
The APL (and generally Federal certification and accreditation) is problematic. FIPS 201 is one part of a regime that solution providers must comply with and can include as examples Federal Information Security Management Act (FISMA), DIACAP/DIARMF (FIPS 199, Controls, SP 800-37, 53A, 137) FIPS 140 Crytpographic Modules, Common Criteria, TWIC (Transportation Worker Identification Credential..). These are all moving and multiple requirements for certification and accreditation. Security and information technology product and system providers in the Federal enterprise have a lot to bear to get into the market. These requirements do not play nicely with rapidly developing technology. Patches, and version upgrades could require new certifications. All of which might be acceptable if you get to a federated and interoperable network effect from the requirements, otherwise it’s a sink hole.
For the APL to matter it needs to provide the Federal
enterprise customers and suppliers a list of items they can buy or build to
construct identity application systems and the supporting infrastructure. The system components should be interoperable
and the APL should provide an ability to test this. The test infrastructure and requirements
should be public facing, documented, be freely available and when possible as open
source. Ideally there would be more of
an option for self compliance. It would
reduce the number and frequency of interactions with national validation
laboratory accreditation program organizations.
In effect the APL should provide a development environment for testing
FICAM applications and infrastructure.
If you think you have something that plays plug it in and see what
happens.
Evolving in this way provides more value to end users and
industry. End users can leverage the
competition among multiple suppliers and move to a common set of procurement
requirements, consolidate training and maintenance. Manufacturers can develop systems and test
them in the course of manufacturing in a way that would meet the test
laboratory and certification requirements but also be something they would do
anyway by mimicking/leveraging the product assurance they perform in making
products generally available. These changes
would better address the goals of HSPD-12, reduce costs and timelines and
improve productivity, privacy and security.
Thanks Sal for this nice analysis and breakdown. In my mind interoperability testing is similar to continuous monitoring wherein products are continuously tested with other sets of products that they interface with. If a new product fails to work in an interoperable manner, then the root cause has to be investigated. This could mean that all previous approved products be retested depending on the results of the failure assessment. Nothing wrong with this model and it at the end of the day provides full assurance that all approved products will be interoperable with the set of products/systems they are expected to work with. However, this requires a test regiment that works on matrix-based testing and probably the EP have to move from a vendor-based cost model to a government-paid model.
ReplyDeleteThanks Nabil, I think its possible to have a hybrid. Government should provide infrastructure, policy and tools and have private sector as national validation laboratory extension. Both need to make an investment in the next step.
ReplyDeleteIsn't this what is currently being done? GSA provides the overall test guidelines and infrastructure and the labs conduct the tests in accordance with GSA's prescribed methodology. That being said, I understand what you are trying to get to - "Product categories that are more aligned with solution components and tests that more aligned with best practices in product assurance and interoperability". For this to happen, I believe the FIPS 201 Evaluation Program has to be given some freedom to go out of the box. In the past, only test cases that were mappable to requirements from FIPS 201 and its supporting publications made it to the approval and test procedures. However it was well known that several other tests could be added that would give greater assurance in the product/system's ability to meet interoperability and performance requirements. Use cases that promote interoperability and best practices in Industry need to be tested for the products/system categories, however identifying and agreeing on these requirements might be challenging. Vendors should be given the ability to be creative and develop their offerings without being too prescriptive. Labs should have access to products/systems that are tested by other labs so as to be able to perform the interoperability tests. Tradeoffs - Short test cycles with limited assurance on interoperability or longer test cycles with greater assurance on interoperability. Self-certification might help in cost and time savings, but when things don't work in the field - that's when things go out of hand and fingers are pointed in all directions !!
ReplyDeleteThe point is that the infrastructure has to represent some aspect of product development, product assurance and use. FIPS 201 focus is identity and authentication token, not really an authorization credential, when the FIPS 201 APL tries to test authorization in this authentication context (not all items are about authorization and therefore don't run into this) it makes sense that it runs into challenges. There are significant costs (or savings) in play here for the Federal government so it would necessarily need to sponsor the activity. Certainly there is an accreditation above self but it can mimic. I appreciate the comments Nabil.
ReplyDeleteThis is a nice post..........Credentialing
ReplyDeleteHere are further details from the GSA
ReplyDeleteJune 7, 2012
Introductions
Purpose:
GSA OGP would like to hear from stakeholders how the FIPS 201 Evaluation Program can evolve and adapt to the changing needs of our Agency Customers
Topics for Discussion:
• Feedback on process and procedures from the Vendor/Solution Providers
• Feedback on process and procedures current approved Labs
• Input from entities that are seeking to become approved labs
• Input on current suite of Test Tools
• Input on value of shared infrastructure to be shared across Labs
• Discussion on orienting T&E towards interoperability in addition to conformance
Way Forward:
• Gather issues identified by Agency Customers, Vendors, NIST, Testing Labs and other Stakeholders
• Performing top to bottom review of the FIPS 201 Test and Evaluation Process
• Update and/or enhance testing tools if needed
• Update process and procedures for T&E
• If a business need exists, explore the feasibility of a common shared infrastructure for Vendors, Labs and Agencies to test their products
• Creating methods for tiered interoperability testing between
components/products
** Dial in numbers: US: 517-386-5253 Toll Free: 1-877-937-0988 Participant Passcode: 9928607
http://blog.idmanagement.gov/2012/06/fips-201-evaluation-program-industry.html follow up from GSA FIPS 201 Evaluation Program
ReplyDeleteThis is great blog. I am pretty much impressed with your good work. You put really very helpful information. Digital Signature Certificate
ReplyDelete