Discharge7443/fleet
1
0
Fork 0
Code Issues Pull Requests Actions 24 Packages Projects Releases Wiki Activity

removing cypress references for e2e and add docs for QA wolf (#10382)

Browse Source 
Documentation updates around cypress and new QA wolf platform for e2e
testing
This commit is contained in:
Gabriel Hernandez
2023-03-08 16:00:57 +00:00
committed by GitHub
parent f031011938
commit 5f30d2eafb
3 changed files with 17 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Contributing/Fleet-UI-Testing.md
+6 -52
View File
@@ -33,7 +33,6 @@ For instructions on using our testings tools, check out our [testing docs](https
- [ESLint and TypeScript](#es-lint-and-typescript)
- [Jest](#jest)
- [Testing library](#testing-library)
- [Cypress](#cypress)
- [Additional examples](#additional-examples)
- [Roles and permissions](#roles-and-permissions)
- [Mac and Windows hosts](#mac-and-windows-hosts)
@@ -125,8 +124,8 @@ We use a variety of testing to ensure that our software is working as intended.
| Reusable utilities | Unit with Jest | Devs | String util, url util | Little to no dependencies. Function argument-based. |
| Reusable hooks | Unit with Jest & react-hooks Library | Devs | useToggleDisplayed Hook | Little to no dependencies. Function argument-based. |
| Reusable UI components | Unit with Jest & testing-library | Devs | Radio, button, and input components | Little to no dependencies. Props-based. |
| App widgets | Integration or E2E with testing-library or Cypress | End users | Create user form. Reset password form. | Less reusable code with more complex environment setup and dependencies. Depending on the case, can be done with integration or E2E. For integration, mock at the backend level; don't mock other UI systems. For E2E, don't mock other systems except for common network error states. |
| User journeys | E2E with Cypress | End users | Filtering a host by software. Creating a team as admin. | Full business flows. Little to no mocking of systems, except for common network error states. |
| App widgets | Integration or E2E with testing-library or QA Wolf | End users | Create user form. Reset password form. | Less reusable code with more complex environment setup and dependencies. Depending on the case, can be done with integration or E2E. For integration, mock at the backend level; don't mock other UI systems. For E2E, don't mock other systems except for common network error states. |
| User journeys | E2E with QA Wolf | End users | Filtering a host by software. Creating a team as admin. | Full business flows. Little to no mocking of systems, except for common network error states. |
| N/A | Manual | N/A | // TODO | Manual testing can be used for all types of code. Examples would be for one-offs or states that would require extremely difficult testing setups. |
#### Manual testing
@@ -285,45 +284,8 @@ describe("ResetPasswordForm - component", () => {
#### E2E testing
Our E2E layer tests all the systems of the software (frontend and backend) together to
ensure the application works as intended. To support this, we rarely mock API responses at this
layer of testing. Exceptions include mocking network error responses and mocking external APIs, such
as Jira or Zendesk integration responses. At this level, we want to test the software as an actual user.
[View the full labels flow user journey testing source](https://github.com/fleetdm/fleet/blob/main/cypress/integration/all/app/labelflow.spec.ts).
```ts
describe("Labels flow", () => {
before(() => {
// ...setup
});
after(() => {
// ...teardown
});
describe("Manage hosts page", () => {
beforeEach(() => {
// ...setup
});
it("creates a custom label", () => {
cy.getAttached(".label-filter-select__control").click();
cy.findByRole("button", { name: /add label/i }).click();
cy.getAttached(".ace_content").type(
"{selectall}{backspace}SELECT * FROM users;"
);
cy.findByLabelText(/name/i).click().type("Show all MAC users");
cy.findByLabelText(/description/i)
.click()
.type("Select all MAC users.");
cy.getAttached(".label-form__form-field--platform > .Select").click();
cy.getAttached(".Select-menu-outer").within(() => {
cy.findByText(/macOS/i).click();
});
cy.findByRole("button", { name: /save label/i }).click();
cy.findByText(/label created/i).should("exist");
});
});
});
```
ensure the application works as intended. We have partnered with QA Wolf to cover these flows, and
the E2E tests are written and maintained by them.
### Tooling
@@ -345,18 +307,10 @@ integration testing.
#### Testing library
We rely heavily on the different libraries that are part of the testing-library ecosystem for our
unit and integration testing. These including react-testing-library, cypress-testing-library,
react-hooks, and user-events. The guiding principles of the testing-library tools align with our own
unit and integration testing. These including react-testing-library, react-hooks, and user-events.
The guiding principles of the testing-library tools align with our own
in that we believe tests should resemble real-world usage as closely as possible.
#### Cypress
We use Cypress with Cypress Testing Library as our E2E testing framework. We primarily rely on full
E2E software testing and rarely mock API responses, but we make exceptions for mocking common
network error responses and testing app integrations with external APIs.
For more details on our E2E testing, check out our [Cypress documentation](https://github.com/fleetdm/fleet/tree/97b4d1f3fb30f7b25991412c0b40327f93cb118c/cypress/README.md).
### Additional examples
#### Roles and permissions
docs/Contributing/Testing-and-local-development.md
+6 -19
View File
@@ -156,26 +156,13 @@ go tool cover -func=coverage.txt
## End-to-end tests
E2E tests are run using Docker and Cypress.
We have partnered with [QA Wolf](https://www.qawolf.com/) to help manage and maintain our E2E testing suite.
The code is deployed and tested once daily on the testing instance.
E2E tests are constantly evolving, and running them or examining CI results is the best way to understand what they cover, but at a high level, they cover:
1. Setup
2. Log in/out flows
3. Host page
Add hosts
Label flows
4. Query flows
5. Policy flows
6. Schedule flows
scheduling
packs
6. Permissions
Admin
Observer (global and team)
Maintainer
7. Organizational Settings
Settings adjustments
Users
QA Wolf manages any issues found from these tests and will raise github issues. Engineers should not
have to worry about working with E2E testing code or raising issues themselves.
For Fleet employees, if you would like access to the QA Wolf platform you can reach out in the [#help-engineering](https://fleetdm.slack.com/archives/C019WG4GH0A) slack channel.
### Preparation
frontend/README.md
+5 -2
View File
@@ -20,10 +20,13 @@ Visit the [overview of Fleet UI testing](../docs/Contributing/Fleet-UI-Testing.m
To run unit or integration tests in `ComponentName.tests.tsx`, run `yarn test -- ComponentName.tests.tsx`. To [test all Javascript components](https://fleetdm.com/docs/contributing/testing-and-local-development#javascript-unit-tests) run `yarn test`.
To run E2E tests, visit [our Cypress testing documentation](../cypress/README.md).
For more information on how our front-end tests work, visit our [frontend test directory](./test/README.md).
[QA Wolf](https://www.qawolf.com/) manages our E2E test and will maintain the tests as well as raise
any issues found from these tests. Engineers should not have to worry about working with E2E testing
code or raising issues themselves.
For more information on how our front-end tests work, visit our [frontend test
directory](./test/README.md).
## Directory structure