Download Beta User Guide - Neighbourhood Statistics

NDE2 Quick Start Guide
NeSS Data Exchange
NeSS Data Exchange
Quick Start
February 2010
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 1 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
1.
Introduction ............................................................................................... 4
2.
Endpoints ................................................................................................... 5
3.
List of Operations ....................................................................................... 6
4.
Core Operations.......................................................................................... 7
4.1
FindAreas([Postcode], [AreaNamePart], [Code], [HierarchyId],
[LevelTypeId]) ............................................................................................. 7
4.2
FindDatasets (Metadata) ....................................................................... 8
4.3
GetAreaChildren (AreaId)...................................................................... 8
4.4
GetAreaDetail (AreaId) ......................................................................... 9
4.5
GetDatasetDetail (DSFamilyId) .............................................................. 9
4.6
GetDatasets (SubjectId, [AreaId]).......................................................... 9
4.7
GetSubjects () ................................................................................... 10
4.8
GetSubjectDetail (SubjectId) ............................................................... 10
4.9
GetSubjectTree () .............................................................................. 10
4.10
GetVariableDetail (VarFamilyId)........................................................ 10
4.11
GetVariables (DSFamilyId, [DateRange]) ........................................... 11
4.12
GetTables (Areas, [Datasets], [Variables], [TimePeriod], [OutputFormat],
[OutputVersion], [GroupByDataset]) ............................................................. 11
4.13
GetChildAreaTables (ParentArea, [LevelTypeId], [Datasets], [Variables],
[TimePeriod], [OutputFormat], [OutputVersion], [GroupByDataset]) .................. 12
5.
Advanced Operations................................................................................ 13
5.1
GetAreaAtLevel (AreaId, LevelTypeId) .................................................. 13
5.2
GetAreaComparators (AreaId) ............................................................. 13
5.3
GetCompatibleSubjects (AreaId) .......................................................... 13
5.4
GetDatasetFamilies (SubjectId, [AreaId]) .............................................. 13
5.5
GetHierarchies () ............................................................................... 14
5.6
GetHierarchyDetail (HierarchyId) ......................................................... 14
5.7
GetLevelTypes () ............................................................................... 14
5.8
GetLevelTypesByHierarchy (HierarchyId)............................................... 14
5.9
GetModifiedDatasetFamilies (Date, [SubjectId, [AreaId]])........................ 15
5.10
GetVariableFamilies (DSFamilyId, [DateRange]).................................. 15
5.11
SearchAreaByCode (Code, [HierarchyId], [LevelTypeId])...................... 15
5.12
SearchAreaByNameHierarchy (AreaNamePart, [HierarchyId])................ 16
5.13
SearchAreaByNameLevelType (AreaNamePart, LevelTypeId)................. 16
5.14
SearchAreaByPostcodeHierarchy (Postcode, [HierarchyId]) ................... 17
5.15
SearchAreaByPostcodeLevelType (Postcode, LevelTypeId) .................... 17
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 2 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
6.
5.16
SearchSByAByName (AreaNamePart, LevelTypeId).............................. 18
5.17
SearchSByAByPostcode (Postcode,[HierarchyId]) ................................ 18
5.18
SearchDatasetByMetadata (Metadata) ............................................... 18
5.19
GetDataCube () POST only .............................................................. 19
Appendix A - Reference ............................................................................ 20
6.1
Hierarchies........................................................................................ 20
6.2
Subjects ........................................................................................... 20
6.3
Level Types....................................................................................... 21
6.4
Measurement Units ............................................................................ 22
6.5
Statistical Units ................................................................................. 23
6.6
Periodicities....................................................................................... 23
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 3 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
1.
Introduction
NeSS Data Exchange (NDE) is a free web service which lets you run queries against the
Neighbourhood Statistics database. This database contains more than 300 datasets
covering a variety of social statistics, held for a number of different geographic areas. It
possible to mix, match and compare these statistics by Area and over time. The database
also contains useful geographic information.
NDE is a complementary service to the web site, and the two share a common database
and common components enabling the user to select, extract and visualise data.
Browser
Website
Data
Access
Component
Data
base
Fig. 1 Website
Browser
AJAX
Flex
Spread
sheet
Excel
Macro
Other
Java
.Net
t
NDE
Web
Service
Data
Access
Component
Data
base
Fig. 2 Web Service
The two diagrams above compare a website with a web service.
Figure 1 shows how the user accesses the NeSS website via his web browser. He can
only view or download data using the screens programmed into the website.
Figure 2 shows how the web service adds both flexibility and automation to the
processes. The web service layer allows applications written by the user to run the same
data access components that the NeSS website uses, by means of simple operations.
NDE2 supports access via both SOAP and REST protocols, both running without
authentication. The previous version of NDE (1.2) is still working but this is SOAP only
and does require a username and password in the SOAP header.
The REST version is not strictly a RESTful service, rather it allows you to call NDE’s
operations via a simple URI, making it more suitable for modern and lightweight
development platforms.
NDE offers a large selection of operations which can be used to construct an application.
These applications are divided into “core” and “advanced” and are described in this
manual (with further details in the Technical Implementation Guide).
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 4 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
The operations are also divided into “discovery” and “delivery” operations, spread over
two endpoint URLs. Discovery’ operations ask questions of the database, such as:
•
•
•
•
‘What are the child areas of Fareham local authority (LA)?’
‘Which Datasets hold data for Fareham LA?’
‘Which variables comprise dataset 57?’
‘Which datasets have been updated since December 2008?’
These all return responses in simple, bespoke XML format which can be serialised into
objects when using SOAP, or interpreted in a textual format with REST. In many cases
the results of one Delivery query are fed into another operation call. This is sometimes
called ‘orchestration’.
Finally, the information from the Discovery calls is used to construct a Delivery request.
This can be achieved using two simple operations or a more complex, flexible documentbased query.
The response to this query is the data ‘payload’ in LGDX format (SDMX and RDF are also
available). See the Technical Implementation Guide for details and sample output. This
response will often then be transformed via an XSLT into a table, but it could just as
easily be processed and stored on a database - or anything else you want to do with it.
The two main types of application are real time access (‘hunter’) and local data stores
(‘gatherer’). NDE is fast enough for real time use - most discovery queries run in well
under a second though delivery queries can take up to 10 seconds when approaching the
cell limit of 10000. This limit does affect local data store applications in that it means
they can only collect data a chunk at a time. Where the data store is for a specific area
such as an LA, this won’t usually be a problem. For national data it is better to use the
download facility on the NeSS website. It should be noted that this facility currently only
gives you loaded data, whereas the web service will give you on-the-fly aggregated data.
NDE no longer requires registration, but you can always get help from NeSS support
([email protected]) and the NDE Users Google Group
(http://groups.google.co.uk/group/ness-data-exchange?hl=en )
2.
Endpoints
The endpoints are:
Discovery: http://neighbourhood.statistics.gov.uk/NDE2/Disco
Delivery: http://neighbourhood.statistics.gov.uk/NDE2/Deli
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 5 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
3.
List of Operations
Table 1 - Discovery Operations
Core
Advanced
GetAreaAtLevel
GetAreaChildren
GetAreaComparators
GetAreaDetail
GetCompatibleSubjects
GetDatasetDetail
GetDatasets
GetSubjects
GetSubjectDetail
GetSubjectTree
GetVariableDetail
GetVariables
FindAreas
FindDatasets
GetDatasetFamilies
GetHierarchies
GetHierarchyDetail
GetLevelTypes
GetLevelTypesByHierarchy
GetModifiedDatasetFamilies
GetVariableFamilies
SearchAreaByCode
SearchAreaByNameHierarchy
SearchAreaByNameLevelType
SearchAreaByPostcodeHierarchy
SearchAreaByPostcodeLevelType
SearchSByAByName
SearchSByAByPostcode
SearchDatasetByMetadata
Table 2 - Delivery Operations
Core
GetTables
GetChildAreaTables
Advanced
GetDataCube
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 6 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
4.
Core Operations
4.1
FindAreas([Postcode], [AreaNamePart], [Code],
[HierarchyId], [LevelTypeId])
This operation is a composite search operation which combines all the existing area
searches. The postcode, area string and SNAC code are separate parameters to avoid
parsing complexities. A setting of HierarchyId=0 tells the service to use “Stats by Area”
rules to pick hierarchy for level type. One of postcode, area name part and (SNAC) Code
must be supplied. All other parameters optional.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?Postcode=PO155RR
Give me a vertical slice of areas containing PO155RR in the default
hierarchy (GPH)
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?Postcode=PO155RR&Hiera
rchyId=4
Give me a vertical slice of areas containing PO155RR in Hierarchy 4
(2003Admin)
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?Postcode=PO155RR&Level
Type=13
Give me the LAs that PO155RR falls in, for all hierarchies (e.g. 2004
version of Fareham, 2005 version of Fareham etc.)
http://212.58.231.244/NDE2/Disco/FindAreas?Postcode=PO155RR&LevelType=13&Hiera
rchyId=0
Give me the LA that PO155RR falls in and use the Stats By Area rule table
to decide which hierarchy to use.
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?Postcode=PO155RR&Level
Type=13&HierarchyId=17
Give me the LA that PO155RR falls in and only give me the version for
Hierarchy 17.
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?AreaNamePart=avo
Give me all areas matching the string ‘avo’ in the GPH (default)
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?AreaNamePart=avo&Hiear
achyId=4
Give me all areas matching the string ‘avo’ in 2004Admin
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?AreaNamePart=avo&Level
Type=13
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 7 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
Give me all areas matching the string ‘avo’ with level type 13 (LA)
regardless of hierarchy
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?AreaNamePart=avo&Level
Type=13&HierarchyId=0
Give me all areas matching the string ‘avo’ with level type 13 (LA) and use
the Stats By Area rule table to choose the hierarchy
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?AreaNamePart=avo&Level
Type=13&HierarchyId=17
Give me all areas matching the string ‘avo’ with level type 13 (LA) and
select only those for hierarchy 17
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindAreas?Code=24UE&HierarchyId=
4
Give me all areas with the SNAC (External) Code of 24UE (exact match) in
the 2004 Admin hierarchy (should return only one area)
4.2
FindDatasets (Metadata)
This operation allows you to perform a text search to find matching datasets. The string
is compared to:
- dataset titles
- keywords held against the dataset
- metadata held for the dataset
- the titles of variables belonging to the dataset
- metadata held for variables belonging to the dataset
The results are always dataset families. There is currently no facilitiy to search for
variables this way.
This is currently the same as advanced operation SearchDatasetByMetadata.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/FindDatasets?Metadata=GCSE
Give me all datasets with the word ‘GCSE’ in the metadata or keyword list
or variable title
4.3
GetAreaChildren (AreaId)
This operation allows you to get the lower level areas for a supplied area, regardless of
level type. Only those areas immediately below the supplied area are returned. For
example, the child areas of a region would be the counties and unitary authorities within
that region.
Repeatedly calling this operation allows you drill down through area hierarchies.
This is all you need for a realtime geography selector (hunter) or to build an offline
hierarchy (gatherer). The concept is simple too - just get the areas that sit within the one
named, one level below (e.g. all the wards in an LA).
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 8 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetAreaChildren?AreaId=276980
Give me all areas which are the the immediate child of Fareham LA (GPH) in this case a list of MSOAs
4.4
GetAreaDetail (AreaId)
This is a simple operation which gives you extra information about an area, by supplying
its AreaId. This AreaId is internal to the NeSS datastore and must have been previously
obtained via other discovery calls.
In the returned information, the Envelope can be used in a GIS system – it is the four
corners of the map of the area MinX:MinY:MaxX:MaxY - these numbers being Ordnance
Survey Eastings and Northings. The ExtCode is the standard SNAC code for the area.
There may be metadata for the area or nil elements.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetAreaDetail?AreaId=276980
Give me the attributes of Fareham LA (GPH) - SNAC code, envelope etc
4.5
GetDatasetDetail (DSFamilyId)
This operation allows you to request the optional metadata associated with your
requested dataset. The information returned is broadly similar to that displayed on the
NeSS website, but slightly changed for machine to machine usage.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetDatasetDetail?DSFamilyId=1163
Give me information about the Fire Incidents dataset.
4.6
GetDatasets (SubjectId, [AreaId])
This operation enables you to obtain the dataset families associated with your requested
subject. The SubjectId will have been obtained from a getSubjects() call. An AreaId is
optional, and if supplied, only datasets which have data for this area will be returned.
This is currently the same as the advanced operation getDatasetFamilies.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetDatasets?SubjectId=10&AreaId=
276980
Give me a list of all datasets with subject ‘Indexes of Deprivation’ which
have data for Fareham LA (GPH).
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 9 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
4.7
GetSubjects ()
This operation enables you to obtain the available subjects, but not their associated
dataset families (for which you use GetDatasetFamilies). This differs from GetSubjectTree
which returns all dataset families for all subjects.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetSubjects?
Give me a list of all subjects on the NeSS Database
4.8
GetSubjectDetail (SubjectId)
This operation allows you to obtain the metadata associated with your requested subject.
Dataset families are not returned.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetSubjectDetail?SubjectId=10
Give me the description for subject 10 (Indexes of Deprivation)
4.9
GetSubjectTree ()
This operation has no parameters and returns a nested tree of subjects by datasets with
supported date ranges for each dataset.
The datasets listed are dataset families each of which has one or more dataset instances,
(see getVariableFamilies for a fuller description of families and instances) each one of
these being shown by a DateRange element. Often the DateRange is a single day on
which the data was collected.
It was originally intended this would be an advanced operation, partly because it takes a
little while to run so is more suited to offline storage. However, testing it with e4x in
Adobe Flex the one-shot tree proved to be very useful - you can put the result into an
Arraycollection and dynamically drive selection components without repeated web service
calls. Note that it runs a lot quicker in 2.0 than 1.2 (no SSL encryption).
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetSubjectTree?
Please retrieve the entire tree structure of datasets within subject.
4.10
GetVariableDetail (VarFamilyId)
This operation allows you to obtain the metadata associated with your requested
variable. The information returned is broadly similar to that displayed on the NeSS
website, but slightly changed for machine to machine usage.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetVariableDetail?VarFamilyId=125
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 10 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
Give me the all the information held for variable 125 (Hours Worked by
Females)
4.11
GetVariables (DSFamilyId, [DateRange])
This operation gives you the variable family ids for a dataset.
In the NeSS Datastore, we use the concept of families and instances. A dataset that is
collected annually would have an instance for each year. So an example of a family
would be ‘Unemployment Statistics’ and its instances ‘Unemployment Statistics 2000’,
‘Unemployment Statistics 2001’ etc.
Within each dataset, we call the constituent items ‘variables’ (or ‘TOPIC’ in LGDX). In our
example the variable families might be ‘Unemployed females’, ‘Unemployed males’ and
‘Unemployed IT developers’. These families would have corresponding variable instances
under all of the dataset instances, except where a variable is added or removed over
time. For example in 2005 ONS decided to no longer count unemployed IT developers.
For each variable, we return its id, name, measurement unit and statistical unit. A
measurement unit is a count, rate, percentage etc. A statistical unit is a person,
household etc. So ‘percentage of households with broadband’ would have the
measurement unit ‘percentage’, and statistical unit ‘household’.
The response includes the dates for which the variable has been used, so it is possible to
filter out variables that are not populated at the date of interest. A date range can also
be specified in the call, so that only variables extant within that date range will be
returned.
This is currently the same as the advanced operation getVariableFamilies
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetVariables?DateRange=2001-0101:2010-12-31&DSFamilyId=1893
Give me a list of variables for dataset 1893, extant between 2001 and 2010.
{NEXT TWO OPERATIONS ARE DELIVERY}
4.12
GetTables (Areas, [Datasets], [Variables], [TimePeriod],
[OutputFormat], [OutputVersion], [GroupByDataset])
This operation is a simplified version of getDataCube which it encapsulates. As well as
making it easier for client applications to construct the request, this simplification is an
essential precursor to offering a RESTful version of the service.
A list of one or more area ids must be supplied (comma delimited)
All other parameters are optional, but the DataCubeQuery validator will complain unless
one or both of datasets and variables are supplied. If datasets but not variables are
supplied, it is assumed that all the variables are wanted for these datasets.
If no time period is given, then the latest data is returned, only the variables still in use
by the latest version of the dataset are displayed, and no discontinued ones.
OutputFormat is no longer ignored. The default is “LGDX” which will be assumed if it is
omitted. The other options are “SDMX” and “RDF” in which case the output will be
converted to SDMX or RDF format as appropriate. Both of these output formats have
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 11 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
provisional status for 1.2, and will be final in 2.0. RDF currently cannot be generated for
pick and mix requests.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Deli/getTables?Areas=308058&Variables=6
845,6846,6847,6848,6849,6850,6851,6852&GroupByDataset=Yes
Give me selected Index of Deprivation data for St Albans 001A
4.13
GetChildAreaTables (ParentArea, [LevelTypeId], [Datasets],
[Variables], [TimePeriod], [OutputFormat],
[OutputVersion], [GroupByDataset])
This operation is a simplified version of getDataCube which it encapsulates. As well as
making it easier for client applications to construct the request, this simplification is an
essential precursor to offering a RESTful version of the service.
GetChildAreaTables works the same way as getTables, except that instead of a list of
areas, a single parent area id is supplied. If no level type id is supplied, it is assumed
that the direct children are required (uses getAreaChildren to fill out the getDataCube
request). If a level type id is specified, you can explicitly state which small area is to be
used (e.g LSOAs in a LA) - getAreaAtLevel is used to build the getDataCube request.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Deli/getChildAreaTables?ParentAreaId=276
980&LevelTypeId=141&Datasets=67
Give me the complete census population dataset for all the LSOAs in Fareham
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 12 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
5.
Advanced Operations
The definitions below are unchanged from the current service.
5.1
GetAreaAtLevel (AreaId, LevelTypeId)
This operation gives you child areas for a known area id. This AreaId is internal to the
NeSS datastore and must have been previously obtained via other discovery calls. Your
application needs a list of level types (see GetLevelTypes). In the example in Annex A,
the start AreaId 276704 is West Midlands region (in the NeSS Geography hierarchy), and
we are asking for type 12 children. Type 12 is County. So in effect we are asking ‘Give
me all the counties in West Midlands’. Use getAreaDetail to get more information on
these areas (if required).
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetAreaAtLevel?LevelTypeId=141&Ar
eaId=276980
5.2
GetAreaComparators (AreaId)
This operation gives you the higher level areas which contain your chosen area, which
are considered useful for comparison purposes. For example, the comparator areas of an
Output Area might be the local authority, the region (if within England) and the country.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetAreaComparators?AreaId=27698
0
5.3
GetCompatibleSubjects (AreaId)
This operation retrieves the available subjects, without dataset families. With each
subject, however, a count is returned of dataset families that are compatible with the
supplied area.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetCompatibleSubjects?AreaId=2769
80
5.4
GetDatasetFamilies (SubjectId, [AreaId])
This operation enables you to obtain the dataset families associated with your requested
subject. The SubjectId will have been obtained from a getSubjects() call. An AreaId is
optional, and if supplied, only datasets which have data for this area will be returned.
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 13 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetDatasetFamilies?SubjectId=7&Are
aId=276980
5.5
GetHierarchies ()
This operation is called without parameters and yields a list of all the Geographic
Hierarchies in the NeSS datastore. This list is not static, new hierarchies are added from
time to time. For an explanation of UK geographic hierarchies go to:
http://www.statistics.gov.uk/geography/beginners_guide.asp
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetHierarchies?
5.6
GetHierarchyDetail (HierarchyId)
This operation allows you to obtain the metadata associated with the hierarchy of your
requested area, including the root area for the hierarchy. The root area id is useful as a
start point for GetAreaChildren(AreaId).
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetHierarchyDetail?HierarchyId=17
5.7
GetLevelTypes ()
This operation is called without parameters and returns a list of current level types on the
NeSS datastore. A level type is a general type of area about which data is held, for
example a Local authority (LA) – a level type can exist in more than one hierarchy, but
there is only one level type per level in each hierarchy.
In the sample response in Annex A, a comment has been added to tell you what the
letters stand for, and those types for which large amounts of data are held are shown in
bold type.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetLevelTypes?
5.8
GetLevelTypesByHierarchy (HierarchyId)
This operation is called with one parameter, the HierarchyId (the operation
getHierarchies gives you a list of these). It returns a list of current level types on the
NeSS datastore belonging to the specified hierarchy. The list is ordered from largest to
smallest area.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetLevelTypesByHierarchy?Hierarchy
Id=18
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 14 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
5.9
GetModifiedDatasetFamilies (Date, [SubjectId, [AreaId]])
This operation retrieves dataset families that have been updated since the supplied date.
If a subject is supplied, only those datasets associated with the subject are considered. If
an area is supplied, only those datasets compatible with the area are considered.
Note that it is not currently possible to detect deletions (except by comparing an old and
new response).
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetLevelTypesByHierarchy?Hierarchy
Id=2
5.10
GetVariableFamilies (DSFamilyId, [DateRange])
This operation gives you the variable family ids for a dataset.
In the NeSS Datastore, we use the concept of families and instances. A dataset that is
collected annually would have an instance for each year. So an example of a family
would be ‘Unemployment Statistics’ and its instances ‘Unemployment Statistics 2000’,
‘Unemployment Statistics 2001’ etc.
Within each dataset, we call the constituent items ‘variables’ (or ‘TOPIC’ in LGDX). In our
example the variable families might be ‘Unemployed females’, ‘Unemployed males’ and
‘Unemployed IT developers’. These families would have corresponding variable instances
under all of the dataset instances, except where a variable is added or removed over
time. For example in 2005 ONS decided to no longer count unemployed IT developers.
For each variable, we return its id, name, measurement unit and statistical unit. A
measurement unit is a count, rate, percentage etc. A statistical unit is a person,
household etc. So ‘percentage of households with broadband’ would have the
measurement unit ‘percentage’, and statistical unit ‘household’.
The response includes the dates for which the variable has been used, so it is possible to
filter out variables that are not populated at the date of interest. A date range can also
be specified in the call, so that only variables extant within that date range will be
returned.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/GetVariableFamilies?DateRange=200
1-01-01:2010-12-31&DSFamilyId=1893
5.11
SearchAreaByCode (Code, [HierarchyId], [LevelTypeId])
The operation getAreaDetail returns the SNAC code of an area (as ExtCode). This allows
you to convert from a NeSS internal area ID to a SNAC code. The
SearchAreaByCodeElement makes the reverse coversion possible - for example a GIS
package might want to use the SNAC code as a key to create an information pop-up
about an area.
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 15 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
Because the NeSS database holds multiple hierarchies, there will usually be more than
one AreaId per SNAC code. When a new administrative hierarchy is issued, if a ward’s
boundary has changed it is allocated a new SNAC code. If the boundary has not changed
it has the same SNAC code but still gets a new AreaId.
This operation can be called with just a code, in which case a list of areas will be
returned. If a hierarchyId is supplied it is almost certain that a single area only will be
found. If the LevelTypeId is also supplied there can only be one result. Supplying a
LevelTypeId and no HiearchyId is also an option and will give multiple results.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/SearchAreaByCode?Code=24UE&Lev
elTypeId=13&HierarchyId=2
5.12
SearchAreaByNameHierarchy (AreaNamePart,
[HierarchyId])
This operation allows you to search for an area in a particular hierarchy. The parameters
are a string which is all or part of the matching area’s name, and the hierarchy id
(obtained from getHierarchies). Some strings such as ‘West’ could give a large number of
results. Large result sets can also be obtained from the NeSS Geography Hierarchy as
LSOAs are named Fareham 001A, Fareham 001B etc. If no match is found for a valid
area string, an empty response is returned.
The area is sent with a ‘falls within’ area attached. This is the area that the rules table
thinks is the most suitable parent or grandparent area, and will not necessarily be in the
same hierarchy. The falls within area can be nil.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/SearchAreaByNameHierarchy?Hierarc
hyId=7&AreaNamePart=Itchen
5.13
SearchAreaByNameLevelType (AreaNamePart, LevelTypeId)
This operation allows you to search for an area with a particular level type. You can get
similar areas from multiple hierarchies (e.g. 2003 and 2004 version of the Admin
hierarchy). The parameters are the level type id (from getLevelTypes) and a string which
is all or part of the matching area’s name. Some strings such as ‘South’ could give a
large number of results. If no match is found for a valid area string, an empty response
is returned.
The area is sent with a ‘falls within’ area attached. This is the area that the rules table
thinks is the most suitable parent or grandparent area, and will not necessarily be in the
same hierarchy. The falls within area can be nil.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/SearchAreaByNameLevelType?LevelT
ypeId=14&AreaNamePart=Castle
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 16 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
5.14
SearchAreaByPostcodeHierarchy (Postcode, [HierarchyId])
This operation allows you to search for an area in a particular hierarchy. The parameters
are a full postcode, and the hierarchy id (obtained from getHierarchies). The result is a
‘vertical stack’ of areas each one being the parent of the previous one.
If no match is found for a valid postcode string, an empty response is returned.
The area is sent with a ‘falls within’ area attached. This is the area that the rules table
thinks is the most suitable parent or grandparent area, and will not necessarily be in the
same hierarchy. The falls within area can be nil.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/SearchAreaByNameLevelType?LevelT
ypeId=27&AreaNamePart=Itchen
5.15
SearchAreaByPostcodeLevelType (Postcode, LevelTypeId)
This operation allows you to search for an area with a particular level type. You can get
similar areas from multiple hierarchies (e.g. 2003 and 2004 version of the Admin
hierarchy). The parameters are the level type id (from getLevelTypes) and a postcode
(complete).
If no match is found for a valid postcode string, an empty response is returned.
The area is sent with a ‘falls within’ area attached. This is the area that the rules table
thinks is the most suitable parent or grandparent area, and will not necessarily be in the
same hierarchy. The falls within area can be nil.
Note that hierarchies with incomplete coverage, such as parishes, do affect the results.
Search by Name for Fareham (LevelType LA) and you get a match in hierarchies 2,3,4,9
and 11. Search by Postcode for PO155RR (LevelType LA) and you get a match in
hierarchies 2,3,4 and 11.
The reason is that although Fareham exists as an LA in the Parish Hierarchy (9), there
are no parishes in Fareham so this ‘breaks the chain’.
Search by Postcode is implemented as
1) Find Postcode
2) Go from Postcode to its Output Area in the NeSS Geography Hierarchy
3) Link from this OA to the smallest area in the Parish Hierarchy (parish)
4) Work up the Parish Hierarchy until you find an object at LevelType LA.
The result is that the LA can only be found if the postcode is within a real parish.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/SearchAreaByPostcodeLevelType?Lev
elTypeId=20&Postcode=PO155RR
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 17 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
5.16
SearchSByAByName (AreaNamePart, LevelTypeId)
The two SByA operations simulate the behaviour of the Statistics By Area facility on the
NeSS web site. In order to speed up and simplify the area selection process, a number of
potential user choices are automated. A table of rules is used to make these choices.
One principal choice is that for each potentially matching area, a selection is made of the
'most appropriate' hierarchy (taking into account time and boundary/name changes).
This saves selecting between e.g. 1998, 2003, 2004 Fareham Ward as the system
chooses the 2004 Fareham Ward as 'most appropriate'.
This operation allows you to supply a name (or part of a name) and all matching areas
with the specified level type are returned. In the example in Annex A we are searching
for LAs (type 13) with the characters ‘york’ as part of the name. If no match is found for
a valid area string, an empty response is returned.
The area is sent with a ‘falls within’ area attached. This is the area that the rules table
thinks is the most suitable parent or grandparent area, and will not necessarily be in the
same hierarchy. The falls within area can be nil.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/SearchSByAByName?LevelTypeId=14
&AreaNamePart=Upper
5.17
SearchSByAByPostcode (Postcode,[HierarchyId])
The two SByA operations simulate the behaviour of the Statistics By Area facility on the
NeSS web site. In order to speed up and simplify the area selection process, a number of
potential user choices are automated. A table of rules is used to make these choices. A
selection is made of the 'most appropriate' hierarchy (taking into account time and
boundary/name changes).
This operation allows you to supply a postcode and a single area, with the specified level
type, is returned. In Annex A we are asking which LA (type 13) postcode PO175JE is in.
If no match is found for a valid postcode string, an empty response is returned.
The area is sent with a ‘falls within’ area attached. This is the area that the rules table
thinks is the most suitable parent or grandparent area, and will not necessarily be in the
same hierarchy. The falls within area can be nil.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/SearchSByAByPostcode?LevelTypeId
=180&Postcode=PO155RR
5.18
SearchDatasetByMetadata (Metadata)
This operation, allows you to perform a text search to find matching datasets. The string
is compared to:
- dataset titles
- keywords held against the dataset
- metadata held for the dataset
- the titles of variables belonging to the dataset
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 18 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
- metadata held for variables belonging to the dataset
The results are always dataset families. There is currently no facilitiy to search for
variables this way.
Sample Calls:
http://neighbourhood.statistics.gov.uk/NDE2/Disco/SearchDatasetByMetadata?Metadata
=pregnancy
{NEXT OPERATION IS DELIVERY}
5.19
GetDataCube () POST only
This is the original query inherited from the original “Cafeteria” prototype. It was
designed for flexibility, being able to cope with data in any format from multiple
repositories. It is likely that most requests can be achieved using the simpler GetTables
and GetChildAreaTables operations, but getDataCube is still available if needed and will
continue to be supported for backward compatibility.
On the query element, if you set the type attribute to “SDMX” the output will be
converted to SDMX format instead of LGDX. Similarly, setting the type attribute to “RDF”
the output will be converted to RDF format instead of LGDX.
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 19 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
6.
Appendix A - Reference
6.1
Hierarchies
2 NeSS Geography Hierarchy
3 1998 Administrative Hierarchy
4 2003 Administrative Hierarchy
7 2003 Electoral Hierarchy
8 2003 Health Hierarchy
9 2003 Parish Hierarchy
10 2003 Education Hierarchy
11 2004 Administrative Hierarchy
12 New Deal for Communities (Best fit)
14 Provisional Parliamentary Constituencies 2007
15 2006 Health Hierarchy
16 2005 Administrative Hierarchy
17 2006 Administrative Hierarchy
18 2007 Administrative Hierarchy
6.2
Subjects
1 Access to Services
2 Community Well-being / Social Environment
3 Crime and Safety
4 Economic Deprivation
5 Education, Skills and Training
6 Health and Care
7 Housing
8 Physical Environment
9 Work Deprivation
10 Indices of Deprivation and Classification
13 People and Society: Population and Migration
14 People and Society: Income and Lifestyles
15 2001 Census: Key Statistics
16 2001 Census: Census Area Statistics
46 Indicators
47 Access to Services
48 Community Well-being / Social Environment
49 Crime and Safety
50 Economic Deprivation
51 Education, Skills and Training
52 Health and Care
53 Housing
54 Physical Environment
55 Work Deprivation
56 Cross Cutting Indicators
57 People and Society
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 20 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
6.3
Level Types
Level Id
164
10
12
180
9
96
21
8
11
19
13
141
140
201
17
15
16
25
22
20
24
23
97
81
143
144
28
7
29
153
14
27
Level Code
COM
CTRY
CTY
EA
EW
FCTY
GAZ
GB
GOR
HA
LA
LSOA
MSOA
NDC
NP
OA
P
PA
PCD
PCO
PD
PS
RO
SHA
SN
SPR
U
UK
USD
USOA
WARD
WPC
Level Name
Community
Country
County
Education Area
England and Wales
Former County
Gazetteer
Great Britain
Government Office Region
Health Authority
Local Authority
Lower Layer Super Output Area
Middle Layer Super Output Area
New Deal for Community
National Park
Output Area
Parish
Postcode Area
Postcode
Primary Care Organisation
Postcode District
Postcode Sector
Regional Office
Strategic Health Authority
Statistical Neighbourhood
Scottish Parliamentary Region
Urban Area
England Scotland Wales Northern Ireland
Urban Subdivisions
Upper Layer Super Output Area
Ward
Westminster Parliamentary
Constituency
Entries in bold are “Stats by Area” types (as used on NeSS web site home page).
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Page 21 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
6.4
Measurement Units
Measurement
Unit Id
1
2
3
4
5
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
36
37
38
39
40
41
42
43
Measurement
Unit Code
%
£
£ (00s)
£k
£M
Y
D
H
min
s
km
mi
m
m2
ha
ac
m2 (000s)
m2
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Measurement Unit Name
Count
Percentage
Rate
Rate per 1000
Rate per 10000
Rank
Pounds Sterling
Pounds Sterling (hundreds)
Pounds Sterling (thousands)
Pounds Sterling (millions)
Score
Years
Days
Hours
Minutes
Seconds
Kilometres
Miles
Metres
Metres squared
Hectares
Acres
None (Reliability Indicator)
Metres squared (thousands)
Ratio
Classification
Rate per 1000 population
Rate per 1000 households
Count (thousands)
Hectares squared
Square metres
Square metres (thousands)
Ranked Group
Percentage Points
Page 22 of 23
Status: Final
NDE2 Quick Start Guide
NeSS Data Exchange
6.5
Statistical Units
Statistical
Unit Id
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
6.6
Statistical Unit name
Persons
Households
Businesses
Enterprise Units
Offences
Household Spaces
Areas
Episodes
Sites
Communal Establishments
Families
Dwellings
Incidents
Cases
Users
Rooms
Local Units
Hereditament
Statistical
Unit Id
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Statistical Unit name
None (Reliability Indicator)
Vehicles
Pollutant
Premises
Live Births
Claims
Convictions
Lettings
Days
Pupil Sessions
Pupil Half Days
Rent
Admission Episodes
Enrolments
Enterprises
Kilowatt Hours
Meter Points
Periodicities
Periodicity
Id
51
54
55
57
129
130
131
175
176
223
224
248
Periodicity
Code
DT
M
CY
FY
CQ
CHY
FHY
RQ
RY
2YRP
3YRP
AY
© Office for National Statistics 2010
Issue No: 0.1 Date: 22/02/2010
~6070384
Periodicity
Name
Date
Month
Calendar Year
Financial Year
Calendar Quarter
Calendar Half Year
Financial Half Year
Rolling Quarter
Rolling Year
2 Year Rolling Period
3 Year Rolling Period
Academic Year
Page 23 of 23
Status: Final