Any chance that you have your rest API available as an OpenAPI file that can be used with a service like swagger.io.
Such a file would allow language bindings to languages other than the few you currently have SDKs for, with Java being my requirement.
Thanks
Roger
@rit001 This is something we’re working on and we hope to have some options available in the relatively near future!
Any update on this, our project is going slowly so I’ve not got to the point of needing to integrate beyond using the CLI, but one day it will become an issue.
@rit001 The official SDK is still in the works, but we do have an OAS file you can access here that might be helpful to you in the mean time:
https://docs.doppler.com/openapi/5dd2498b6b2edc00fbb3a7e3
It’s auto-generated from our API documentation.
Thanks, I’m aware of the API reference page and do check to see how things are changing over time.
Currently what is available seems well written from the point of view of what is needed by your in-house team to work with the API, it lacks descriptive detail in places (but your team members are already experts) - the content of responses is a good example as a reader of the API has to work out what the different return values relate to.
The published API is also showing its age as I think certain aspects of it have been extended without the docs being updated. The 2 examples I can give at the moment are
The service token reference that is shown by the CLI and web GUI. I’ve already posted about the limitations of how the reference is generated from the created token key, but currently, the docs do not indicate that this reference is returned. Instead, the UUID/slug is the only unique reference, but that can not be matched to active tokens.
There is currently no end point documented that allows the IP access control list assigned to service tokens to be accessed/updated. I am guessing that as this is a server-side feature no one has considered it a requirement for the public API, but being able to access this is as important as accessing the defined service tokens once someone needs to pass an IT audit and if, via the API I can create service tokens I also need a way to create their IP access values.
@rit001 Sorry, I guess I’m misunderstanding what you’re after here. Your original post in this thread was asking if we had an OpenAPI file available. You then followed up asking if there was an update, to which I sent you a link to an OpenAPI spec file you could potentially use with swagger.io.
What you’re talking about in your last response seems to be a separate issue? Are you seeing discrepancies with how the API is documented and how it actually works in practice somewhere? The points you mention seem more to be functionality/feature gaps in the API, which shouldn’t prevent you from using the OAS spec file for the API endpoints we have available now.
As far as your individual examples:
Regards,
-Joel
Sorry for throwing so many things into one reply, but I’m currently handling so many different things here and currently Doppler is the foundation for many of them as I am using Doppler as the core info store of all aspects of the IT platform, deployment, and operations.
The API
Firstly, I can not just take the output you provided and import it into swagger.io as you have your API marked as OpenAPI 3.1, while swagger.io currently only supports up to 3.0. This seems fixable by just changing the value to 3.0. I guess the question for your development team is - are they currently using any 3.1 features or have they just set that value as it is the latest version and they were not aware of it causing any issues? If the latter, it may be worth them considering downgrading to a lower version as that currently has better support at swagger.io - or noting the fact somewhere that this can be changed so that an import into swagger.io is possible. See the end note as setting a version lower than 3.0 may also be helpful.
After more experimentation, my comment about the lack of detail regarding the API, seems to be an issue with your generated API web pages. The tool you are using does not expose any of the schema info held within the Openapi definition, instead, it just shows the examples. The schema info is rather useful as it defines the type of a value, including enum lists.
While the OpenAPI file does seem to define the schema detail for the request, this is not always true for the response, instead, just the example text is provided for some endpoints.
The way in which endpoints have been named is problematic when using a third-party tool like swagger.io. This may be just a limitation of not having access to a 3.1 parser, but currently, the tool you use to generate the API docs seems to infer more from the json definition than swagger.io can when it comes to naming and grouping. This can be seen via the endpoints /v3/projects (GET and POST), they are in fact /v3/projects/list and /v3/projects/create, but the distinction is handled by the HTTP verb and not the endpoint name. Your tool infers the list/create aspect of the endpoint, from the summary attribute, swagger.io does not.
Your web documentation tool groups endpoints so all endpoints starting /v3/projects/ are grouped together (but not sorted). swagger.io does not group or sort, instead, it shows a list based on the order found in the underlying .json file.
So the API definition is work in progress (which is fine) but the real question is how much focus can Doppler currently place on advancing the API. Once you have a cleanly defined 3.1 (it has to be 3.1, see below) schema you will be able to create a basic language wrapper for most programming languages, until then it is hard to build automation around Doppler.
Note on why OpenAPI v3.1 is set - it seems that the OpenAPI project team has gotten itself into a bit of a mess. Until v3.0 it was valid to pass a request body when calling an endpoint defined with the delete verb and the Doppler API makes use of this feature. For V3.0 it was decided to stop a request body from being passed, but for v3.1 it will again be allowed.
Service Token naming
The issue I’m currently facing is that they is no consistent human-readable form of name used across the Doppler environment for a Token. This is rather key for issue resolution and auditing.
At the moment there are
At the time of creation, I know
context - The known config path - workplace/project/environment/config
name - The text name given to a token
Token - dp.st.dev.b0kg0ZVodCQDF0kL3U0hFI4vhSJdHC6XN3p34ZukXIg
When reading a Token I know
Token - dp.st.dev.b0kg0ZVodCQDF0kL3U0hFI4vhSJdHC6XN3p34ZukXIg
config - dev
When a Token is passed to Doppler CLI
context - The known config path - workplace/project/environment/config
Short form - dp.st…ZukXIg
When I look at the list of Tokens defined against a config via the web GUI
context - The known config path - workplace/project/environment/config
name - The text name given to a token
When I look at the list of Tokens defined via the Tokens page in the web GUI
context - The known config path - workplace/project/environment/config
name - The text name given to a token
Short form - dp.st…ZukXIg
When I look at the Logs of a config
context - The known config path - workplace/project/environment/config
name - The text name given to a token
When I access tokens via the API
context - The known config path - workplace/project/environment/config
name - The text name given to a token
UUID - The Doppler database unique ID of the token
So there is no common way to work with a token from an OPS or security point of view.
The full token name does not provide the detail needed to find the config it is defined in, while the short form shown by the CLI and the Token web GUI page can not be used/accessed via the API. The fact that the short form uses the last few digits from the token’s random portion, rather than the first few digits also makes things harder as I have to regex-based searches rather than simple text searches for keys.
The rather simple tasks I need to deal with include
The current API does not help with any of the above, instead, I would need to screen scrape the Token page in the web GUI as my starting point. The reason for this is that it only provides an internal UUID as a reference and I am hoping that the UUID is not related to the token’s value in any computable way.
All of the above simple tasks lead to a longer-term task - that of rotating tokens over time.
IP access control API’s
Can you please take this as a request to add me to the list of people asking for this to be exposed by the API.
The background is rather simple, I am using Doppler in a cloud environment that must conform to the EU rules regarding the Payment Card Industry Data Security Standards, The Data Protection Law Enforcement Directive, as well as expectations set by payment platforms.
Being able to clearly define that only certain systems can access certain secrets is rather key and the IP-based controls are currently the only real solution. Without this API interface, I can not do the following
Automate any of my Doppler work as I can not create the IP-secured tokens via code.
Audit my configuration. With this API it would be possible to scan for incorrect configurations - such as 0.0.0.0/0 still being allowed or an unexpected IP address/range having been defined against a config. Currently, the only way to do an audit is to look at each config in turn via the web GUI, which is not a real solution.
@rit001 Thanks for all your feedback in this thread! In particular, the OpenAPI feedback is helpful. Just to clarify though, that OpenAPI spec was 100% generated by Readme.io (the service we use for our docs) and is made from the manually crafted documentation for the API there. So, what you get is 100% generated by what’s shown in the API documentation. I’ll pass your feedback along in case there’s something we can do to make improvements. That said, we are working on providing some SDKs that will ultimately be (mostly) generated via the OpenAPI spec, so as that progresses we’ll be fixing things that do cause problems. Feedback from someone else who actually tried using it is great though!
Re: Service Token Naming
I see your frustration here and this is something we definitely want to solve. We have an internal issue open for improving our Tokens page significantly which should solve this long term. In the near term, we’re working on an API endpoint that is currently still evolving (the docs are updated to reflect a change going out in the next deploy that adds more detail to what this endpoint returns). It still doesn’t address all of what you’re after (e.g., it doesn’t provide the project or config a service token is for), but that’s something we plan to improve in another pass (essentially, we’re still determining what metadata we can/should provide for various token types). Minimally though, you’ll be able to hit this endpoint with a token and get the token slug, preview, name, type, and workplace (along with a couple other things) once this deploy goes out (likely later today or early Monday). Hopefully, in the next few weeks we’ll be able to expand this a bit further to make it solve most of your issues when it comes to identifying tokens.
Re: IP Access Control API
Thanks for your feedback on this! I’ll definitely pass it along. Providing your specific use case was very helpful!
Regards,
-Joel
Thanks for the detailed reply. There is nothing I have raised that I’m dependent on for many months and the need to be able to handle audits is even further away, so it is good to know that you are least looking at these things for the future.
The fact that you are using readme.io explains a few of the issues I listed - such as the use of OpenAPI 3.1 for the output - it is the default of the site. The larger issue of not being able to use your output OpenAPI spec is also explained as readme.io is great at producing human readable docs, but at least with the enter-level version which I have looked at does not provide any validation tools. I was taking the output and placing it into swagger.io to generate code based on the spec file. So the html based examples given for some of the responses work OK for documentation, but a site like swagger.io needs the schema detail.
Some additional feedback.
Your ‘me’ endpoint has 3 major issues if the token is being used in a secured environment that needs to be audited as the token is used for authentication
As the token is being used to collect data regarding itself, it means the audit log and last used date will be polluted by any attempt to find out details about a token.
If IP address restrictions have been placed on a token this interface is not going to help track down enough detail about a token to be able to find out what IP address restrictions have been placed on a token! (I hope that makes sense) as the only system(s) able to call the endpoint will be those listed in the IP address list.
A discovered token should not really give up details about itself, which I think is why your last post had a focus on limiting what info is presented by this endpoint. The problem is any info is too much info, so please do not add more return data. Having this endpoint just makes me want to put in place even more IP list restrictions on my environment. Over time I would also not be surprised to hear that you end up with requests to disable this endpoint from large clients that do full security audits.
In many ways, your current ‘me’ endpoint needs an ‘inspect’ counterpart for security/audit work that is accessed via your user/permissions security layers, rather than the token layer. The upside for you as a business would be that such endpoints (including a possible future IP list endpoint) would become ‘value add’ features for Team and Enterprise accounts or maybe even just Enterprise.
