42 Commits
Author SHA1 Message Date
Martin Angers d249aa888b Android: add 'Android' builtin label, add new Android hosts to 'All hosts' and this label (#26585) 2025-02-25 14:44:48 -05:00
Tim Lee 7b86f1ee6f Add Linux encryption states to APIs (#23806) 2024-11-19 13:11:59 -07:00
Tim Lee 98c0bd8d12 Add Fedora built in label (#22465) 2024-09-30 10:02:54 -06:00
Lucas Manuel Rodriguez fcdda20664 Backend for policy automation to install software (#21650)
#21428

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
2024-08-30 14:13:25 -03:00
Lucas Manuel Rodriguez 28ca463d13 iOS/iPadOS as platforms/labels (#20126)
#19963 

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality

---

# API changes for dashboard UI changes

## Main dashboard page

`GET /api/latest/fleet/host_summary?low_disk_space=32` (see
`ios`/`ipados` platforms and `iOS`/`iPadOS` labels)
```json
{
  "totals_hosts_count": 9,
  "online_count": 0,
  "offline_count": 9,
  "mia_count": 0,
  "missing_30_days_count": 0,
  "new_count": 0,
  "all_linux_count": 2,
  "low_disk_space_count": 3,
  "builtin_labels": [
    {
      "id": 1,
      "name": "macOS 14+ (Sonoma+)",
      "description": "macOS hosts with version 14 and above",
      "label_type": "builtin"
    },
    {
      "id": 7,
      "name": "All Hosts",
      "description": "All hosts which have enrolled in Fleet",
      "label_type": "builtin"
    },
    {
      "id": 8,
      "name": "macOS",
      "description": "All macOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 9,
      "name": "Ubuntu Linux",
      "description": "All Ubuntu hosts",
      "label_type": "builtin"
    },
    {
      "id": 10,
      "name": "CentOS Linux",
      "description": "All CentOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 11,
      "name": "MS Windows",
      "description": "All Windows hosts",
      "label_type": "builtin"
    },
    {
      "id": 12,
      "name": "Red Hat Linux",
      "description": "All Red Hat Enterprise Linux hosts",
      "label_type": "builtin"
    },
    {
      "id": 13,
      "name": "All Linux",
      "description": "All Linux distributions",
      "label_type": "builtin"
    },
    {
      "id": 14,
      "name": "chrome",
      "description": "All Chrome hosts",
      "label_type": "builtin"
    },
    {
      "id": 15,
      "name": "iOS",
      "description": "All iOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 16,
      "name": "iPadOS",
      "description": "All iPadOS hosts",
      "label_type": "builtin"
    }
  ],
  "platforms": [
    {
      "platform": "darwin",
      "hosts_count": 3
    },
    {
      "platform": "ios",
      "hosts_count": 1
    },
    {
      "platform": "ipados",
      "hosts_count": 1
    },
    {
      "platform": "rhel",
      "hosts_count": 1
    },
    {
      "platform": "ubuntu",
      "hosts_count": 1
    },
    {
      "platform": "windows",
      "hosts_count": 2
    }
  ]
}
```

## After selecting a platform

`GET /api/latest/fleet/host_summary?platform=ios&low_disk_space=100`
(similar with `ipados`)
```json
{
  "totals_hosts_count": 1,
  "online_count": 0,
  "offline_count": 1,
  "mia_count": 0,
  "missing_30_days_count": 0,
  "new_count": 0,
  "all_linux_count": 0,
  "low_disk_space_count": 1,
  "builtin_labels": [
    {
      "id": 1,
      "name": "macOS 14+ (Sonoma+)",
      "description": "macOS hosts with version 14 and above",
      "label_type": "builtin"
    },
    {
      "id": 7,
      "name": "All Hosts",
      "description": "All hosts which have enrolled in Fleet",
      "label_type": "builtin"
    },
    {
      "id": 8,
      "name": "macOS",
      "description": "All macOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 9,
      "name": "Ubuntu Linux",
      "description": "All Ubuntu hosts",
      "label_type": "builtin"
    },
    {
      "id": 10,
      "name": "CentOS Linux",
      "description": "All CentOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 11,
      "name": "MS Windows",
      "description": "All Windows hosts",
      "label_type": "builtin"
    },
    {
      "id": 12,
      "name": "Red Hat Linux",
      "description": "All Red Hat Enterprise Linux hosts",
      "label_type": "builtin"
    },
    {
      "id": 13,
      "name": "All Linux",
      "description": "All Linux distributions",
      "label_type": "builtin"
    },
    {
      "id": 14,
      "name": "chrome",
      "description": "All Chrome hosts",
      "label_type": "builtin"
    },
    {
      "id": 15,
      "name": "iOS",
      "description": "All iOS hosts",
      "label_type": "builtin"
    },
    {
      "id": 16,
      "name": "iPadOS",
      "description": "All iPadOS hosts",
      "label_type": "builtin"
    }
  ],
  "platforms": [
    {
      "platform": "ios",
      "hosts_count": 1
    }
  ]
}
```

### To populate list of MDM solutions of a selected platform

`GET /api/latest/fleet/hosts/summary/mdm\?platform=ios` (similar with
`ipados`)

```json
{
  "counts_updated_at": "2024-06-27T21:56:45Z",
  "mobile_device_management_enrollment_status": {
    "enrolled_manual_hosts_count": 0,
    "enrolled_automated_hosts_count": 1,
    "pending_hosts_count": 0,
    "unenrolled_hosts_count": 0,
    "hosts_count": 1
  },
  "mobile_device_management_solution": [
    {
      "id": 1,
      "name": "Fleet",
      "server_url": "https://lucas-fleet.ngrok.app/mdm/apple/mdm",
      "hosts_count": 1
    }
  ]
}
```

### To populate OS versions of a selected platform

`GET /api/latest/fleet/os_versions?platform=ipados` (similar with `ios`)
```json
{
  "meta": {
    "has_next_results": false,
    "has_previous_results": false
  },
  "count": 1,
  "counts_updated_at": "2024-06-27T21:36:12Z",
  "os_versions": [
    {
      "os_version_id": 7,
      "hosts_count": 1,
      "name": "iPadOS 17.5.1",
      "name_only": "iPadOS",
      "version": "17.5.1",
      "platform": "ipados",
      "vulnerabilities": []
    }
  ]
}
```

## Filtering hosts by the two new `iOS`/`iPadOS` labels

Works the same as with other labels.
2024-07-08 18:05:29 -03:00
gillespi314 3b3f815a42 Merge conflicts 2024-04-16 10:20:59 -05:00
Martin Angers 47279ca2a9 Fix conflicts 2024-04-15 16:10:10 -04:00
Sarah Gillespie 05ccf9ee23 Fix issues related to Fleet builtin labels and reserved MDM profile names (#18043) 2024-04-08 14:34:55 -05:00
Lucas Manuel Rodriguez e58e72fb77 Prevent empty logging_type when creating and editing queries (#14575)
#14551

- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2023-10-16 19:33:39 -03:00
Lucas Manuel Rodriguez 2afbd24021 Combine Schedules and Queries: API changes (#12778)
Combining schedules and queries API changes.
2023-07-24 20:17:20 -04:00
Juan Fernandez 6a69604c62 Updated factory method for creating queries in tests 2023-07-07 09:05:51 -04:00
gillespi314 94dd1c3745 Ingest pending MDM hosts (#9065)
Co-authored-by @roperzh
2022-12-26 15:32:39 -06:00
Frank Sievertsen baa1ddc0f2 Add MDM detection for windows and mdm endpoints (#8479) 2022-11-01 18:22:07 +01:00
Frank Sievertsen e9f7066d87 7135 host display name (#7873) 2022-10-08 08:57:46 -04:00
Martin Angers 9a0f749641 Add hosts_count field to "list software" endpoint (#3873) 2022-01-26 09:47:56 -05:00
Lucas Manuel Rodriguez e64a88d8b1 Add COALESCEs on all host_seen_times JOINs (#3147)
* Add COALESCEs on all host_seen_times JOINs

* Use tx instead of d.writer

* Add unit tests

* Fix compile test
2021-12-01 09:05:23 -03:00
Lucas Manuel Rodriguez cb54d9a8dc Fix duplicate schedules and platform matching on scheduled queries (#2977)
* Fix duplicate schedules and platform matching on scheduled queries

* scheduled_queries.platform can be NULL

* Add unit tests

* Add rhel host and check zero stats
2021-11-17 19:03:30 -03:00
Tomas Touceda 2033d8208c Add policy updated at (#2246)
* wip

* Add policy updated at interval and update the UI to use that

* Update rest api

* Fix tests
2021-09-27 16:27:38 -03:00
Martin Angers 4f4185372d Add support for context in datastore/mysql layer (#1962)
This is just to pass down the context to the datastore layer, it doesn't
use it just yet - this will be in a follow-up PR.
2021-09-14 08:11:07 -04:00
Tomas Touceda 7c34956d31 Add coverage to uncovered mysql code (#1855)
* Add coverage to uncovered mysql code

* Add deleted method and update mock

* Fix test
2021-09-07 13:48:04 -03:00
Tomas Touceda c6c63ab12a Refactor app config (POC, for now) (#1685) 2021-08-20 12:27:41 -03:00
Tomas Touceda 29570bd860 Issue 1278 select leader (#1367)
* Add leader selection

* remove comment

* Address review comments

* Add changes file

* Simplify implementation

* Simplify further

* Whoops, removed a little too much
2021-07-19 15:08:41 -03:00
Tomas Touceda 322ac3c8f6 Make roles for users mandatory (#1338)
* Make roles for users mandatory

* Remove nop migration

* Add missing test for wrong role

* Properly validate global and team roles

* Address codacy issues

* Address codacy review

* No need to check for nil
2021-07-13 16:33:04 -03:00
Zach Wasserman c5280c0517 Add v4 suffix in go.mod (#1224) 2021-06-25 21:46:51 -07:00
RachelElysia aeb852e168 Remove username from UI (#1168)
* Remove username from UI code
* Remove username from tests
* Remove username from database
* Modify server endpoints for removing username
* Implement backend aspects of removing username
* Update API docs
* Add name to fleetctl
2021-06-24 13:42:29 -07:00
Zach Wasserman 1417d01407 Make naming of host columns consistent (#1183)
Adding consistency between API and DB helps to make it easier for users
and developers working with the API to correctly order things.

Closes #317
2021-06-23 17:32:19 -07:00
Zach Wasserman fb32f0cf40 Remove kolide types and packages from backend (#974)
Generally renamed `kolide` -> `fleet`
2021-06-06 15:07:29 -07:00
Noah Talerman 72882e8f9f Merge branch 'master' into teams 2021-05-19 13:16:54 -04:00
Zach Wasserman f788254e61 Implement storage of scheduled query statistics (#735)
Track all data from the osquery_schedule table on a per-host basis. This
data is now returned when retrieving host details in the API.
2021-05-06 21:05:09 -07:00
Zach Wasserman b12a6cb4c1 Migrate old admin field to new global role (#609)
- Migrate old admins to global admins
- Migrate old non-admins to global maintainers
- Remove old admin column
- Give initial user global admin privilege
- Comment out some tests (to be refactored for new permissions model later)
2021-04-06 18:27:10 -07:00
Mike Arpaia af96e52a00 Update the Go import paths to new repo name (#27) 2020-11-11 09:59:12 -08:00
Zachary Wasserman 42bea2a144 Implement manual labels
"Manual" labels can be specified by hostname, allowing users to specify
the membership of a label without having to use a dynamic query. See the
included documentation.
2020-07-21 14:05:46 -07:00
Zachary Wasserman 608772917c Refactor label membership storage
Label membership is now stored in the label_membership table. This is
done in preparation for adding "manual" labels, as previously label
membership was associated directly with label query executions.

Label queries are now all executed at the same time, rather than on
separate intervals. This simplifies the calculation of which distributed
queries need to be run when a host checks in.
2020-07-21 14:05:46 -07:00
Zachary Wasserman 0502412e15 Move live query operations from MySQL to Redis
This change optimizes live queries by pushing the computation of query
targets to the creation time of the query, and efficiently caching the
targets in Redis. This results in a huge performance improvement at both
steady-state, and when running live queries.

- Live queries are stored using a bitfield in Redis, and takes
advantage of bitfield operations to be extremely efficient.

- Only run Redis live query test when REDIS_TEST is set in environment

- Ensure that live queries are only sent to hosts when there is a client
listening for results. Addresses an existing issue in Fleet along with
appropriate cleanup for the refactored live query backend.
2020-07-21 14:05:46 -07:00
Zachary Wasserman be6a16959a Fix error messages related to null users as creators of queries. (#1928)
Ensure that the queries page still loads even when users are deleted manually
in the DB.

Fixes #1911
2018-10-11 13:22:43 -07:00
Zachary Wasserman 0b7747bef0 Fix pack and query UI issues in Fleet 2.0 (#1829)
Replaces (and appropriately refactors) a number of endpoints that were removed long ago when we decided to kill the UI with the fleetctl release. We turned out not to do this, and now need to restore these missing endpoints.

This is not a straight up replacement of the existing code because of refactoring to the DB schemas that was also done in the migration.

Most of the replaced code was removed in #1670 and #1686.

Fixes #1811, fixes #1810
2018-06-15 10:13:11 -04:00
Zachary Wasserman bde7256a09 Update labels interfaces for fleetctl (#1686)
- Remove unnecessary labels code
- Remove unnecessary packs code
- Update tests as appropriate
2018-01-10 11:38:20 -08:00
Zachary Wasserman 26dc30bd25 Update query and pack interfaces for fleetctl (#1670)
- Add new Apply spec methods for queries and packs
- Remove now extraneous datastore/service methods
- Remove import service (unused, and had many dependencies that this breaks)
- Refactor tests as appropriate
2018-01-03 11:18:05 -08:00
Mike Arpaia 018b91ab2c Rename project to Kolide Fleet (#1529) 2017-06-22 15:50:45 -04:00
Mike Arpaia a000751bfe renaming kolide-ose to kolide (#1143) 2017-02-01 10:20:50 -07:00
Mike Arpaia 704ddd424b Host summary endpoint (#742)
* Initial scaffolding of the host summary endpoint

* inmem datastore implementation of GenerateHostStatusStatistics

* HostSummary docstring

* changing the url of the host summary endpoint

* datastore tests for GenerateHostStatusStatistics

* MySQL datastore implementation of GenerateHostStatusStatistics

* <= and >= to catch exact time edge case

* removing clock interface method

* lowercase error wraps

* removin superfluous whitespace

* use updated_at

* adding a seen_at column to the hosts table

* moving the update of seen_time to the caller

* using db.Get instead of db.Select
2017-01-04 14:16:17 -07:00
Mike Arpaia f109b14f9d Moving query attributes from the query object to the pack-query relationship (#559)
* Moving query attributes from the query object to the pack-query relationship

* some additional tests

* http request parsing test

* QueryOptions in new test_util code

* initial scaffolding of new request structures

* service and datastore

* test outline

* l2 merge conflict scrub

* service tests for scheduled query service

* service and datastore tests

* most endpoints and transports

* order of values are not deterministic with inmem

* transport tests

* rename PackQuery to ScheduledQuery

* removing existing implementation of adding queries to packs

* accounting for the new argument to NewQuery

* fix alignment in sql query

* removing underscore

* add removed to the datastore

* removed differential from the schema
2016-12-13 14:22:05 -08:00