Repository navigation
feat(cloud): connect AWS, Azure and Google Cloud in a few clicks, and see their resources - #5136
Merged
Merged
Conversation
…grations, connectors, read loop Cloud connections read a cloud account the way an OpenTelemetry Collector's cloud receivers would, with nothing to run: read-only access the customer creates with one command, an hourly inventory (Azure) or a sweep of where resources report (AWS, Google Cloud), and the key metrics of each resource through the metrics ingest in the receivers' own shape, so resources land under Cloud > All Resources with their metrics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the connector tests The Connect page offers AWS, Azure and Google Cloud as tiles; each opens a guided form: run one command in the cloud's own browser terminal, paste what it printed, test, connect. Connected accounts are listed with their status and their resources by service. Tests cover the catalog, the paste reader, the setup commands, costs, key metrics, each cloud's connector against a fake API, the service and the read loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ler, sync and migration tests Counts are plurals in the reader's language, numbers are written their way, and the catalog's access note is extracted with the rest of its words. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e settings; worker, ingest and Dashboard tests A page per cloud's one command, what OneUptime reads and what it costs, held to the code by CloudConnectDocsFacts. Cloud Resources points to it. cloudConnect.aws in the chart (and CLOUD_CONNECT_AWS_* in config.env) give a self-hosted OneUptime the AWS identity that signs in to customers' roles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…idth, dark and Japanese; dark theme guard; formatting The offline suite (packages/E2E/CloudConnect, npm run test-cloud-connect-ui) renders the real page against an in-page API: the tiles, a cloud connected from its tile with its command, one paste and a test, an AWS role or an access key, the list's status and actions. It runs in the UI regressions' group 8. Each tile's Connect now sits on one line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| this.router.post( | ||
| `${basePath}/test`, | ||
| UserMiddleware.getUserMiddleware, |
Comment on lines
+69
to
+147
| async (req: ExpressRequest, res: ExpressResponse, next: NextFunction) => { | ||
| try { | ||
| const props: DatabaseCommonInteractionProps = | ||
| await CommonAPI.getDatabaseCommonInteractionProps(req); | ||
| const projectId: ObjectID = | ||
| CommonAPI.assertAuthenticatedProjectMember(props); | ||
| this.assertCanOperate(props); | ||
|
|
||
| const body: JSONObject = | ||
| req.body && typeof req.body === "object" && !Array.isArray(req.body) | ||
| ? (req.body as JSONObject) | ||
| : {}; | ||
|
|
||
| let settings: CloudConnectionSettings; | ||
|
|
||
| try { | ||
| settings = await this.readSettingsToTest({ | ||
| body, | ||
| projectId, | ||
| props, | ||
| }); | ||
| } catch (error) { | ||
| /* | ||
| * Settings the form could still fix (a role ARN of the wrong | ||
| * shape, a tenant that is not a GUID) are the test's answer, | ||
| * shown next to the field, not a failed request. | ||
| */ | ||
| if (!(error instanceof BadDataException)) { | ||
| throw error; | ||
| } | ||
|
|
||
| const result: CloudConnectionTestResult = { | ||
| provider: String(body["provider"] || ""), | ||
| status: "fail", | ||
| message: error.message, | ||
| failureKind: | ||
| "settings" as CloudConnectionTestResult["failureKind"], | ||
| durationMs: 0, | ||
| }; | ||
|
|
||
| return Response.sendJsonObjectResponse( | ||
| req, | ||
| res, | ||
| result as unknown as JSONObject, | ||
| ); | ||
| } | ||
|
|
||
| const result: CloudConnectionTestResult = | ||
| await CloudConnectorRegistry.getConnector(settings.provider) | ||
| .testConnection(settings, { | ||
| transport: DataSourceHttpFetch.fetch.bind(DataSourceHttpFetch), | ||
| }) | ||
| .catch((error: unknown): CloudConnectionTestResult => { | ||
| // testConnection never throws; this is the belt to its braces. | ||
| const failure: LlmProviderError = LlmProviderError.from({ | ||
| error, | ||
| step: "test", | ||
| providerTitle: settings.provider, | ||
| secretValues: getCloudSecretValues(settings), | ||
| }); | ||
|
|
||
| return { | ||
| provider: settings.provider, | ||
| status: "fail", | ||
| message: failure.message, | ||
| failureKind: failure.kind, | ||
| durationMs: 0, | ||
| }; | ||
| }); | ||
|
|
||
| return Response.sendJsonObjectResponse( | ||
| req, | ||
| res, | ||
| result as unknown as JSONObject, | ||
| ); | ||
| } catch (error) { | ||
| next(error); | ||
| } | ||
| }, |
|
|
||
| this.router.post( | ||
| `${basePath}/:connectionId/read-now`, | ||
| UserMiddleware.getUserMiddleware, |
Comment on lines
+154
to
+194
| async (req: ExpressRequest, res: ExpressResponse, next: NextFunction) => { | ||
| try { | ||
| const props: DatabaseCommonInteractionProps = | ||
| await CommonAPI.getDatabaseCommonInteractionProps(req); | ||
| const projectId: ObjectID = | ||
| CommonAPI.assertAuthenticatedProjectMember(props); | ||
| this.assertCanOperate(props); | ||
|
|
||
| const connectionId: string | undefined = req.params["connectionId"]; | ||
|
|
||
| if (!connectionId || !ObjectID.isValidUUID(connectionId)) { | ||
| throw new BadDataException("A valid connection ID is required."); | ||
| } | ||
|
|
||
| const connection: CloudConnection | null = | ||
| await this.service.findOneBy({ | ||
| query: { _id: connectionId, projectId }, | ||
| select: { _id: true, projectId: true, isEnabled: true }, | ||
| props, | ||
| }); | ||
| CommonAPI.assertResourceBelongsToProject({ | ||
| resourceProjectId: connection?.projectId, | ||
| projectId, | ||
| }); | ||
|
|
||
| if (!connection?.isEnabled) { | ||
| throw new BadDataException( | ||
| "This connection is turned off. Turn it on to read it.", | ||
| ); | ||
| } | ||
|
|
||
| await CloudConnectionScheduler.enqueue( | ||
| new ObjectID(connectionId), | ||
| new Date(), | ||
| ); | ||
|
|
||
| return Response.sendJsonObjectResponse(req, res, { queued: true }); | ||
| } catch (error) { | ||
| next(error); | ||
| } | ||
| }, |
|
|
||
| this.router.get( | ||
| `${basePath}/aws-role-setup`, | ||
| UserMiddleware.getUserMiddleware, |
Comment on lines
+201
to
+223
| async (req: ExpressRequest, res: ExpressResponse, next: NextFunction) => { | ||
| try { | ||
| const props: DatabaseCommonInteractionProps = | ||
| await CommonAPI.getDatabaseCommonInteractionProps(req); | ||
| const projectId: ObjectID = | ||
| CommonAPI.assertAuthenticatedProjectMember(props); | ||
| this.assertCanOperate(props); | ||
|
|
||
| const setup: CloudConnectionAwsRoleSetup = | ||
| await AwsServerIdentity.getRoleSetup({ | ||
| projectId: projectId.toString(), | ||
| transport: DataSourceHttpFetch.fetch.bind(DataSourceHttpFetch), | ||
| }); | ||
|
|
||
| return Response.sendJsonObjectResponse( | ||
| req, | ||
| res, | ||
| setup as unknown as JSONObject, | ||
| ); | ||
| } catch (error) { | ||
| next(error); | ||
| } | ||
| }, |
…t-ui-review-2 # Conflicts: # packages/App/FeatureSet/Dashboard/src/Locales/da.json # packages/App/FeatureSet/Dashboard/src/Locales/de.json # packages/App/FeatureSet/Dashboard/src/Locales/en.json # packages/App/FeatureSet/Dashboard/src/Locales/es.json # packages/App/FeatureSet/Dashboard/src/Locales/fa.json # packages/App/FeatureSet/Dashboard/src/Locales/fr.json # packages/App/FeatureSet/Dashboard/src/Locales/hi.json # packages/App/FeatureSet/Dashboard/src/Locales/it.json # packages/App/FeatureSet/Dashboard/src/Locales/ja.json # packages/App/FeatureSet/Dashboard/src/Locales/ko.json # packages/App/FeatureSet/Dashboard/src/Locales/nl.json # packages/App/FeatureSet/Dashboard/src/Locales/no.json # packages/App/FeatureSet/Dashboard/src/Locales/pt.json # packages/App/FeatureSet/Dashboard/src/Locales/ru.json # packages/App/FeatureSet/Dashboard/src/Locales/sv.json # packages/App/FeatureSet/Dashboard/src/Locales/zh-CN.json # packages/App/FeatureSet/Dashboard/src/Locales/zh-TW.json
…f, and watches a read it starts - The connected accounts list is three columns (Account, Status, Resources): the status carries when it was last read and how often, the resources how many - the way to them - and where. Before, the Last read column ran under the sticky actions at 1280px (German: hidden) and the service chips made rows tall. - Read now no longer asks people to refresh the list: the row says "Reading now..." and the list loads again by itself once the read ends. - The tiles and promises stack below xl: at 768-1279px beside the side menu three columns left one word a line. - The Test step runs the test by itself as it opens; a missing setting is said in words, not only in a greyed button's tooltip. - A saved connection's Test is a dialog of its own (Test again beside Close), every cloud and way in takes one paste, the step says once why it cannot go on, and accounts read are named in the cloud's own word (first UI review's work, salvaged). - The AWS role command names the role and its trust policy once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… language; the docs say the test runs by itself - Read now is locked where it is while its row reads, rather than hidden: the next action no longer slides under the finger on a phone. - The review's new words (the card's sentence, the paste's messages, the accounts in each cloud's word, "Last read", "Not read yet", "Reading now") in all 16 other Dashboard languages; six keys nothing draws any more are dropped. - Connect a Cloud Account (17 languages): each cloud's last step says the form tests the connection as the step opens, and the cost line names the test instead of a button that is no longer pressed. - The browser suite pins the new flow: no test before the Test step, the answer without a press, Read now watched to its end, and the list fitting a 1280 window in English and German with the tiles stacked at 768 and 1024. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and a row's name is not said twice
- A connection waiting for its first read is watched quietly, once, the way
Read now watches a read: when the read ends, the list loads again and the
row shows what it found - nothing to refresh after Connect.
- A name that already says its cloud ("Microsoft Azure", the name a new
Azure connection gets, or "Google Cloud acme-prod") is not followed by the
cloud again: the line under it says only what it reads, with the whole
line on hover.
- An edit's Test step says nothing changes until it is saved (not "until
you connect"), in every language.
- Browser suite: state=new, a just-connected subscription whose row fills
in on its own when its first read ends.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r writes them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ad runs A locked action gives up the row's button to the first usable one, so a read in progress handed its place to Test and the next tap could open a test. View error and Read now are the row's button by placement: View error while the connection fails, else Read now, locked with 'Reading now...' while its read runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 21, 2026
Merged
simlarsen
pushed a commit
that referenced
this pull request
Oct 21, 2026
Cloud Connect (#5136) reached master first with migrations 1805400000000 and 1805450000000: the host collector migrations are renumbered after them (1805500000000, 1805550000000: files, classes, names, Index.ts and their test, which now holds them after Cloud Connect's). Group 8 of the Dashboard Offline UI runs both suites. Locales merged key by key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The ask
A customer monitoring AWS, Azure and Google Cloud keeps telling us OneUptime is hard to connect to his clouds: it takes an OpenTelemetry Collector configuration. The maintainer asked for something as simple as a button that connects Azure, so people can see and monitor their cloud resources.
What this adds: Cloud > Connect
A new first item in the Cloud menu. Pick a cloud from its tile and a three-step form walks you through it:
After that, OneUptime reads the account on a schedule: every 10 minutes for AWS and every 5 for Azure and Google Cloud (5, 10, 15, 30 or 60 to choose from). Its resources show up under Cloud > All Resources with their key metrics, and they can be charted and alerted on with metric monitors like any other metric. The list of connected accounts shows how each one is doing (working, not working, off, waiting for its first read), what it holds, and when it was last read. It has Read now, Test, View error (the error explained the way a test explains it), Turn off/on, Edit and Delete. The empty All Resources page now leads with the same three tiles, and the collector guide is folded under "Already run an OpenTelemetry Collector?".
The approach, and why
aws_cloudwatch,azure_monitor,googlecloudmonitoring) into the same metrics ingest a collector sends to. The existing resource resolver, Cloud Resources pages, metric monitors, drop filters, pipeline rules and retention all apply unchanged. A resource found through a connection and the same resource sent by a collector are one Cloud Resource, not two. A test runs the real ingest to show this for all three clouds.cloudwatch:GetMetricData,cloudwatch:ListMetrics,tag:GetResources). It trusts OneUptime's AWS account under an external ID derived from the project, so a role ARN copied from another project fails to assume. Nothing secret is stored. When an installation has no AWS identity of its own (CLOUD_CONNECT_AWS_ACCESS_KEY_ID/..._SECRET_ACCESS_KEY, orcloudConnect.awsin the chart), the form offers a read-only access key instead. Self-hosted installs work either way.secretsorpullState), and kept on edit unless a new one is typed. They are stripped from every error. Only project owners and admins can connect, test or read now.Existing customers
Nothing changes for anyone sending cloud metrics through a collector. Migrations
1805400000000(a newCloudConnectiontable, plus a nullableCloudResource.cloudConnectionIdwith no default, so no table rewrite) and1805450000000(that column's index and foreign key, built online outside a transaction) both come after master's latest,1805170000000. A resource's overview links to the connection that found it.Left out, for later
Docs
The new page Connect a Cloud Account (
/docs/telemetry/cloud-accounts) is in all 17 languages. It shows each cloud's command word for word, what OneUptime reads and how often, the prices, and troubleshooting, including the Google Cloud organization policy that blocks service account keys.CloudConnectDocsFactsholds it to the code. Cloud Resources links to it.Tests
cloud-connect_test.yaml.packages/E2E/CloudConnect(npm run test-cloud-connect-ui) on the real page with an in-page API, at 1280 and 390 pixels wide, in the dark theme and in Japanese. It runs in the UI regressions' group 8.Verification
After merging master (
8fe0a920f9; the one commit master gained since,dd307a98db, changes only another E2E suite):TestSendRoutesAskTheRuleandSchemaMigrationsOnlineDdl: 648 tests pass.npx jest Guard) with the Cloud page suites andPageSearchIndex: 159 suites pass.McpAuthorizePage, run alongside, passes (125 tests) onceHOSTis set, as CI sets it; without it, one of its tests readshttp:///apion any branch.ee/, the way CI's Common and App jobs run: Common, the guards with this change's suites and the Cloud page suites, 186 suites and 5,730 tests pass; App, the same 14 suites and 3,416 tests pass.npm run test-cloud-connect-ui, 22 tests pass at 1280 and 390 pixels wide, light, dark and Japanese.helm unittest, 49 suites and 612 tests pass (master changed no chart file since).tsc --listFilesshows Common reads only Common files, and the App only App and Common files.prettier --checkand ESLint pass on every changed file (docker-compose.base.ymlis left as master formats it; ESLint's config skips.jsfiles).UI review
What I looked at. Every surface on the real page in its real Cloud layout (side menu, breadcrumbs), drawn by
packages/E2E/CloudConnectwith the Inter font, plus the All Resources page frompackages/E2E/InfrastructureFirstRunand the docs page rendered by the docs' own routes:Widths 1280, 1024, 768 and 390 px; light and dark; English, German and Japanese.
What I changed, and why
The connected accounts list fits. With four columns, the Last read column ran under the sticky actions at 1280 px. In German it was hidden, and the service chips made every row tall. It now has three columns:
It fits 1280 px in English and German, and phones now show Resources too. What a read found by service is still in the test answer, and All Resources lists the resources themselves.
Read now no longer asks people to refresh the list. The row says "Reading now…". Read now stays the row's button but is locked while the read runs. Before, it gave its place to Test, so a second tap opened a test. The list asks the one connection every 5 s and loads again once its last read time moves, so the table does not flicker. It gives up after 7 minutes, since a read stops at 6.
A just-connected account fills in by itself. A connection waiting for its first read is watched the same way, quietly and once. When the read ends, its row shows what it found.
The tiles and the three promises stack below 1280 px (Tailwind
xl). Beside the side menu at 768 to 1279 px, three columns left one word per line.The Test step runs the test as it opens. There is nothing to press, and Test again runs it again. A missing setting is now written under the button, because a greyed button's tooltip never shows on a phone. In an edit, the step says nothing changes until you save. Connect a Cloud Account is updated in all 17 languages: each cloud's last step no longer says "Select Test connection".
The AWS role command names the role and its trust policy once (
ROLE=…,TRUST='…'). Before, it wrote the trust JSON twice, so it now reads top to bottom.Kept from the first UI review round (salvaged after it was stopped):
i18n: 11 new Dashboard strings (14 keys with their plural forms), translated in all 16 other languages. Six keys that nothing draws any more are dropped.
Tests added or changed
Common
CloudConnectionsTable:Common
CloudConnectComponents: the tile grid only splits into columns atxl. The test panel starts itself only once its step is open, never while a setting is missing, and never twice.Common
CloudConnectionFormModal: nothing is tested before the Test step. The test then runs by itself, and Test again runs it again. An edit's test body and wording are checked too.Common
CloudConnectWords: the region names and the last-read wording.Common
CloudConnectionSetupCommands: the five-line role command, its quoting, and that the trust policy appears once.E2E
CloudConnect, now 36 tests (35 run, 1 desktop-only skip on phone). New ones:?state=new) fills in when its first read ends.The suite takes about 1.7 minutes locally, about 1 minute more than before. Its CI group (Dashboard Offline UI 8) is still the lightest.
Verification after merging master
27191bee5fcompile-testsand E2E all type-check.tsc --listFiles: no package reads another's files.Cloud(the connectors, service, API, migrations, catalog, paste readers and the Dashboard Cloud Connect suites): 44 suites, 1424 tests. Master'sCloudResourceMonitoredSweepsPostgres(12 tests) skips itself without a database;npx jest Guard: 155 suites, 4847 tests;Locale: 34 suites;Locale: 10 suites;ee/: the changed suites pass (Common 10 suites, App 8 suites).i18n:checkis sound. Prettier and ESLint are clean on every changed file; the only ESLint output is the known project-service parse error on E2E.jsfixtures. The ReDoS checker finds 0 patterns that are not proven linear.Left as it is
🤖 Generated with Claude Code