5.4.1 DocumentSearchRequest Element Require

onboarding step by step process these steps outline the process to onboard a customer to surescripts data solutions, from account setup through data downloads ăndarăt note this is early adopter documentation requirements and content are subject to change step 1 verify access to data package builder in workbench for instructions on how to set up access, see the setting up access to workbench & data package builder docid\ ck hmjvq8nl6rymbeywf9 instructions step 2 generate keys step 2a generate rsa key pairs for data package builder authentication to authenticate with the data package builder api, you need to generate a 2048 bit rsa key pair and upload the public key through the client keys interface option 1 using open ssl (recommended works on linux, macos, windows) generate a 2048 bit rsa private key openssl genrsa out private key pem 2048 extract the public key openssl req new x509 key privatekey pem out publickey509 pem subj '/cn=data package builder' verify your keys pairs \# view the private key cat privatekey pem \# view the public key cat publickey509 pem option 2 using powershell (windows) generate rsa key pair $rsa = \[system security cryptography rsa] create(2048) export public key in x 509 subjectpublickeyinfo format $publickey = $rsa exportsubjectpublickeyinfo() $publicpem = " begin public key `n" + \[convert] tobase64string($publickey, 'insertlinebreaks') + "`n end public key " $publicpem | out file filepath "public key pem" encoding ascii export private key $privatekey = $rsa exportrsaprivatekey() $privatepem = " begin rsa private key `n" + \[convert] tobase64string($privatekey, 'insertlinebreaks') + "`n end rsa private key " $privatepem | out file filepath "private key pem" encoding ascii write host "keys generated successfully!" write host "public key is in x 509 subjectpublickeyinfo format" what you'll get after generation of the key pair, you will have two files private key pem your private key \ critical keep this secure \ store it in a secure location (hsm, key vault, encrypted storage) \ this will be used to sign a compact jwt to request an acess token from the authorization server public key pem your public key \ this is what you upload to data package builder \ safe to share (it's called "public" for a reason) \ example format \ begin public key miibijanbgkqhkig9w0baqefaaocaq8amiibcgkcaqea0z3vs5jjcds3 \ end public key step 2b upload your public key log in to data package builder navigate to client keys paste the entire contents of public key pem (including the begin/end lines) click create client note this will create the client id that the customer will later use in build the compact jwt assertion docid\ twxbsuhc9a0s7xiehdo11 section ⚠️ important once you have created your client id , notify your surescripts point of contact surescripts must configure your data package before you can proceed to the next steps you will not be able to continue until this configuration is complete security best practices ✅ do generate keys in a secure environment store private keys in a key vault or hsm for production use use strong file permissions on private key files (chmod 600 private key pem on unix) rotate keys periodically ❌ don't share or commit private keys to version control email private keys store private keys in plaintext in application code reuse the same key pair across multiple clients/applications step 2c common errors the below table includes information on common errors that can be encountered when creating key pairs error troubleshooting information "invalid certificate/public key format" error ensure you're uploading the public key, not the private key verify the pem headers are included check for extra whitespace or line breaks key appears to upload but authentication fails verify the corresponding private key is being used for signing requests ensure the key is either 2048 or 4096 (1024 is not allowed and larger key sizes will not fit in our 8 kb jwt input limit) step 3 query builder the query builder in workbench helps you create an odata query url that defines exactly which prescribed medication data you want to retrieve from the data solutions api you can build a query visually or edit it directly, depending on your preference note the query builder is optional; experienced users may submit manual odata queries directly odata is not the only type of request that is supported for additional information on other uses for the query builder, see query builder overview docid\ ufqsg3e7hzr3byccrtwhz for additional information on odata, see https //learn microsoft com/en us/odata/overview https //learn microsoft com/en us/odata/overview important before you begin you must be able to sign in to workbench your user must have access to data package builder with the query edit role you must be working with the prescribed medication product in data solutions for instructions on this, see setting up access to workbench & data package builder docid\ ck hmjvq8nl6rymbeywf9 step 3a run the query builder (odata) sign in to workbench and open data package builder select query builder choose the prescribed medication product select the columns you want returned (these map to $select in odata) add filters to limit results a partition day filter is required select generate from selections to automatically build the odata query (or manually edit the query if you’re familiar with odata) copy the generated odata query url use the url in your data solutions api get request to retrieve prescribed medication data note for information on how to run the query builder for other use cases, see key steps (applies to all use cases) docid\ ufqsg3e7hzr3byccrtwhz step 4 determining partition day filter to determine which partition day filter to request, see the how to use the endpoints docid 1xvta747xdt6ztxvibdlj page step 5 send rest requests to data solutions api authentication learn how to authenticate with the data solutions api using oauth 2 0 client credentials grant with jwt bearer assertion what you'll do the data solutions api uses a two step authentication process build a jwt assertion create and sign a jwt claim using your clientid and private key request an access token exchange the jwt assertion for an oauth access token at the auth server's /connect/token endpoint use the access token include the access token in api requests data solutions authentication process ┌─────────────┐ ┌─────────────┐ │ client │ │ auth server │ └──────┬──────┘ └──────┬──────┘ │ │ │ 1 build jwt assertion │ │ (signed with private key) │ │ │ │ 2 post /connect/token │ │ (client assertion = jwt) │ ├─────────────────────────────────────────────>│ │ │ │ 3 access token response │ │<─────────────────────────────────────────────┤ │ │ │ │ │ ┌──────────────────┴─────┐ │ │ data solutions api │ │ └──────────────────┬─────┘ │ │ │ 4 api request with access token │ ├─────────────────────────────────────────────>│ │ │ │ 5 data response │ │<─────────────────────────────────────────────┤ │ │ before you begin make sure you have client id a unique identifier for your application private key an rsa private key in pkcs#8 pem format public key registration your public key must be registered with the auth server for your client id private key format your private key must be in pkcs#8 pem format \ begin private key miievgibadanbgkqhkig9w0baqefaascbkgwggskageaaoibaqc base64 encoded key data \ end private key step 5a build the compact jwt assertion create a jwt with the following structure, signed with your private key using the rs384 algorithm (rsassa pkcs1 v1 5 with sha 384) this jwt proves your identity to the auth server jwt header { "alg" "rs384", "typ" "jwt" } jwt payload { "iss" "your client id", // issuer your client id "sub" "your client id", // subject your client id "aud" "environment auth url/connect/token", // audience token endpoint url "jti" "550e8400 e29b 41d4 a716 446655440000", // jwt id unique identifier (uuid) "iat" 1706572800, // issued at current unix timestamp "nbf" 1706572800, // not before current unix timestamp "exp" 1706573100 // expires current timestamp + 5 minutes } important field notes iss and sub must both be your client id aud must be the exact token endpoint url (including /connect/token ) jti must be unique for each request (use a uuid) exp should be 5 minutes from iat (recommended maximum) example building jwt in python import jwt import uuid import time from datetime import datetime, timedelta \# your credentials client id = "your client id" private key = """ begin private key miievgibadanbgkqhkig9w0baqefaascbkgwggskageaaoibaqc \ end private key """ token endpoint = "environment auth url/connect/token" \# build the jwt payload now = int(time time()) payload = { "iss" client id, "sub" client id, "aud" token endpoint, "jti" str(uuid uuid4()), "iat" now, "nbf" now, "exp" now + 300 # 5 minutes from now } \# sign the jwt with rs384 client assertion = jwt encode( payload, private key, algorithm="rs384" ) print(f"jwt assertion {client assertion}") example building jwt in c# using system identitymodel tokens jwt; using system security claims; using system security cryptography; using microsoft identitymodel tokens; // your credentials var clientid = "your client id"; var privatekeypem = @" begin private key miievgibadanbgkqhkig9w0baqefaascbkgwggskageaaoibaqc \ end private key "; var tokenendpoint = "environment auth url/connect/token"; // load the private key using var rsa = rsa create(); rsa importfrompem(privatekeypem); var signingcredentials = new signingcredentials( new rsasecuritykey(rsa), securityalgorithms rsasha384 ); // build the jwt var now = datetime utcnow; var handler = new jwtsecuritytokenhandler(); var token = handler createjwtsecuritytoken( issuer clientid, audience tokenendpoint, subject new claimsidentity(new\[] { new claim("sub", clientid) }), notbefore now, expires now\ addminutes(5), issuedat now, signingcredentials signingcredentials ); // add the jti claim token payload\["jti"] = guid newguid() tostring(); var clientassertion = handler writetoken(token); console writeline($"jwt assertion {clientassertion}"); example building jwt in javascript/node js const jwt = require('jsonwebtoken'); const { v4 uuidv4 } = require('uuid'); const fs = require('fs'); // your credentials const clientid = 'your client id'; const privatekey = fs readfilesync('private key pem', 'utf8'); const tokenendpoint = 'environment auth url/connect/token'; // build the jwt payload const now = math floor(date now() / 1000); const payload = { iss clientid, sub clientid, aud tokenendpoint, jti uuidv4(), iat now, nbf now, exp now + 300 // 5 minutes }; // sign the jwt with rs384 const clientassertion = jwt sign(payload, privatekey, { algorithm 'rs384' }); console log(`jwt assertion ${clientassertion}`); step 5b request access token send a post request to the auth server's /connect/token endpoint with the jwt assertion request format post /connect/token http/1 1 host environment auth url content type application/x www form urlencoded grant type=client credentials \&client id=your client id \&client assertion type=urn\ ietf\ params\ oauth\ client assertion type\ jwt bearer \&client assertion=\<your signed jwt> \&scope=dsaapi parameters grant type must be client credentials client id your client identifier client assertion type must be urn\ ietf\ params\ oauth\ client assertion type\ jwt bearer client assertion the signed jwt from step 1 scope must be dsaapi to access the data solutions api example request token in python import requests \# build the token request token url = "environment auth url/connect/token" data = { "grant type" "client credentials", "client id" client id, "client assertion type" "urn\ ietf\ params\ oauth\ client assertion type\ jwt bearer", "client assertion" client assertion, "scope" "dsaapi" } \# request the access token response = requests post(token url, data=data) response raise for status() token response = response json() access token = token response\["access token"] expires in = token response\["expires in"] print(f"access token {access token}") print(f"expires in {expires in} seconds") example request token in c# using system net http; var httpclient = new httpclient(); var tokenurl = "environment auth url/connect/token"; var requestdata = new dictionary\<string, string> { { "grant type", "client credentials" }, { "client id", clientid }, { "client assertion type", "urn\ ietf\ params\ oauth\ client assertion type\ jwt bearer" }, { "client assertion", clientassertion }, { "scope", "dsaapi" } }; var response = await httpclient postasync(tokenurl, new formurlencodedcontent(requestdata)); response ensuresuccessstatuscode(); var tokenresponse = await response content readfromjsonasync\<tokenresponse>(); var accesstoken = tokenresponse accesstoken; var expiresin = tokenresponse expiresin; console writeline($"access token {accesstoken}"); console writeline($"expires in {expiresin} seconds"); // token response model public class tokenresponse { \[jsonpropertyname("access token")] public string accesstoken { get; set; } \[jsonpropertyname("expires in")] public int expiresin { get; set; } \[jsonpropertyname("token type")] public string tokentype { get; set; } } example request token in javascript/node js const axios = require('axios'); const qs = require('querystring'); const tokenurl = 'environment auth url/connect/token'; const requestdata = { grant type 'client credentials', client id clientid, client assertion type 'urn\ ietf\ params\ oauth\ client assertion type\ jwt bearer', client assertion clientassertion, scope 'dsaapi' }; axios post(tokenurl, qs stringify(requestdata), { headers { 'content type' 'application/x www form urlencoded' } }) then(response => { const accesstoken = response data access token; const expiresin = response data expires in; console log(`access token ${accesstoken}`); console log(`expires in ${expiresin} seconds`); }) catch(error => { console error('token request failed ', error response? data || error message); }); successful response { "access token" "eyjhbgcioijsuzm4ncisimtpzci6ijeymzq1njc4otailcj0exaioijkv1qifq ", "expires in" 3600, "token type" "bearer", "scope" "dsaapi" } error response { "error" "invalid client", "error description" "invalid client or client credentials" } common error codes invalid client client id not found or public key mismatch invalid grant jwt assertion validation failed invalid scope requested scope not available step 5c use access token in data solutions api requests include the access token in the authorization header of your api requests request format get /api/v0/odata/prescribedmedicationdata?$filter=partition day eq '2025 01 30' http/1 1 host environment data solutions url authorization bearer \<your access token> example api request in python import requests api url = "environment data solutions url/api/v0/odata/prescribedmedicationdata" headers = { "authorization" f"bearer {access token}" } params = { "$filter" "partition day eq '2025 01 30'", "$top" 10 } response = requests get(api url, headers=headers, params=params) response raise for status() data = response json() print(f"retrieved {len(data\['value'])} records") example api request in c# var apiurl = "environment data solutions url/api/v0/odata/prescribedmedicationdata"; var httpclient = new httpclient(); httpclient defaultrequestheaders authorization = new authenticationheadervalue("bearer", accesstoken); var queryparams = new dictionary\<string, string> { { "$filter", "partition day eq '2025 01 30'" }, { "$top", "10" } }; var querystring = string join("&", queryparams select(kvp => $"{uri escapedatastring(kvp key)}={uri escapedatastring(kvp value)}")); var requesturl = $"{apiurl}?{querystring}"; var response = await httpclient getasync(requesturl); response ensuresuccessstatuscode(); var data = await response content readfromjsonasync\<odataresponse>(); console writeline($"retrieved {data value count} records"); example api request in javascript/node js const axios = require('axios'); const apiurl = 'environment data solutions url/api/v0/odata/prescribedmedicationdata'; const headers = { 'authorization' `bearer ${accesstoken}` }; const params = { '$filter' "partition day eq '2025 01 30'", '$top' 10 }; axios get(apiurl, { headers, params }) then(response => { console log(`retrieved ${response data value length} records`); }) catch(error => { console error('api request failed ', error response? data || error message); }); step 5d handle token expiration access tokens expire after the time specified in the expires in field (typically 3600 seconds / 1 hour) best practices store the token and its expiration time check if the token is expired before each request request a new token when the current one expires or is about to expire add a buffer (e g , refresh 5 minutes before actual expiration) example token management in python from datetime import datetime, timedelta class tokenmanager def init (self, client id, private key, token endpoint) self client id = client id self private key = private key self token endpoint = token endpoint self access token = none self expires at = none def get access token(self) \# return cached token if still valid if self access token and datetime utcnow() < self expires at return self access token \# request new token client assertion = self build jwt() response = requests post(self token endpoint, data={ "grant type" "client credentials", "client id" self client id, "client assertion type" "urn\ ietf\ params\ oauth\ client assertion type\ jwt bearer", "client assertion" client assertion, "scope" "dsaapi" }) response raise for status() token response = response json() self access token = token response\["access token"] \# set expiration with 5 minute buffer expires in = token response\["expires in"] self expires at = datetime utcnow() + timedelta(seconds=expires in 300) return self access token def build jwt(self) \# jwt building logic from step 1 now = int(time time()) payload = { "iss" self client id, "sub" self client id, "aud" self token endpoint, "jti" str(uuid uuid4()), "iat" now, "nbf" now, "exp" now + 300 } return jwt encode(payload, self private key, algorithm="rs384") \# usage token manager = tokenmanager(client id, private key, token endpoint) access token = token manager get access token() optional use dpop for enhanced security the api supports dpop (demonstration of proof of possession) for enhanced security dpop binds access tokens to a specific key pair, preventing token theft dpop token request include a dpop proof jwt in the dpop header post /connect/token http/1 1 host auth server surescripts net content type application/x www form urlencoded dpop \<dpop proof jwt> grant type=client credentials \&client id=your client id \&client assertion type=urn\ ietf\ params\ oauth\ client assertion type\ jwt bearer \&client assertion=\<your signed jwt> \&scope=dsaapi \&dpop jkt=\<jwk thumbprint> dpop proof structure header { "typ" "dpop+jwt", "alg" "rs256", "jwk" { "kty" "rsa", "n" "\<base64url modulus>", "e" "\<base64url exponent>" } } payload { "htu" "environment auth url/connect/token", "htm" "post", "iat" 1706572800, "jti" "unique id here" } dpop api request when using a dpop bound token, include the dpop proof in api requests get /api/v0/odata/prescribedmedicationdata http/1 1 host environment data solutions url authorization dpop \<access token> dpop \<dpop proof jwt> the dpop proof for resource requests must include the ath claim (access token hash) { "htu" "environment data solutions url/api/v0/odata/prescribedmedicationdata", "htm" "get", "iat" 1706572800, "jti" "unique id here", "ath" "\<base64url sha256 of access token>" } for detailed dpop implementation, see the postman collection readme troubleshooting "invalid client" error cause client id not found or public key mismatch solutions verify your client id is correct ensure your public key is registered with the auth server confirm the private key matches the registered public key "invalid grant" error cause jwt assertion validation failed solutions check jwt expiration ( exp claim) ensure it's in the future verify aud claim matches the exact token endpoint url ensure iss and sub both contain your client id confirm jwt is signed with rs384 algorithm check that jwt jti is unique 401 unauthorized on api requests cause invalid or expired access token solutions verify the token hasn't expired check that you're using bearer token type ensure the token was requested with scope=dsaapi confirm the authorization header format bearer \<token> 403 forbidden on api requests cause valid token but insufficient permissions solutions verify your client has purchased the product/package check if the columns you're selecting are in your package ensure you're including a valid partition day filter review package filter restrictions token request timeout cause network connectivity or server issues solutions check network connectivity to the auth server verify the auth server url is correct check firewall rules allow outbound https environment urls environment auth server url data solutions api url production https //auth server surescripts net https //data solutions api surescripts net security best practices protect private keys never commit private keys to source control store keys securely (e g , environment variables, key vaults) use different keys for different environments token storage store tokens securely in memory or encrypted storage never log or expose tokens in error messages clear tokens when they expire jwt claims always use unique jti values (uuids) keep expiration times short (5 minutes recommended) validate token expiration before use https only always use https for token requests and api calls verify ssl/tls certificates error handling implement retry logic for transient failures log authentication failures for monitoring don't expose sensitive details in client facing errors step 6 download data files (optional formats) the data solutions api supports optional file download formats that may be enabled for your data package available formats depend on your package configuration supported formats csv – human‑readable format for simple downloads and manual review parquet – binary, columnar format optimized for analytics and large‑scale processing note file format availability is controlled at the package level by surescripts if a format is not enabled for your package, it is not available for download before you begin confirm that your client id is configured with a package that supports the desired file format you have completed authentication in step 5 you have a valid partition day value file format selection does not affect query construction or authentication download csv data (for enabled csv packages only) send a request to the following endpoint to download csv data using a signed url the signed url is active for 5 minutes and is only available is you are enabled for csv downloads prescribed medication data csv download endpoint /api/v0/customer/product/prescribedmedicationdata/csv/signedurls/{partition day} download parquet data (for enabled parquet packages only) send a request to the following endpoint to download parquet data using a signed url the signed url is active for 5 minutes and is only available is you are enabled for parquet downloads prescribed medication data parquet download endpoint /api/v0/customer/product/prescribedmedicationdata/parquet/signedurls/{partition day}