Object Detection Datasets Beyond COCO

The short answer

Pick by what COCO is missing for your task. If you need more categories, Open Images V7 (600 boxable classes, nearly 16M boxes) or Objects365 (365 categories, 2M images, 30M boxes) both go well beyond COCO's 80. If you need better coverage of rare, long-tailed categories, use LVIS. If your application is autonomous driving, use BDD100K or nuScenes instead of everyday-object photos. If your application is satellite or aerial imagery, use xView, which COCO does not cover at all.

Broader category coverage and scale

Open Images V7

Open Images, maintained by Google, is one of the largest publicly available annotated image collections: version 7 includes 15,851,536 bounding boxes across 600 classes for object detection, 2,785,498 instance segmentation masks across 350 classes, over 3.2 million visual relationship annotations, and more than 61 million image-level labels across over 20,000 classes, with an extension adding 478,000 crowdsourced images and 6,000+ additional classes. Its scale and category breadth make it useful when COCO's 80 categories are simply too narrow for your task, though the label distribution is uneven across classes. Review the specific license for images and annotations on the download pages before commercial use. Get it from storage.googleapis.com/openimages/web.

Objects365

Objects365, released by Megvii and the Beijing Academy of Artificial Intelligence, targets a middle ground between COCO's small, well-curated category set and Open Images' very large but uneven one: 365 object categories across 2 million images with 30 million bounding boxes, built specifically to have more images per category than Open Images so each class is better represented for training. It has become a common large-scale pretraining dataset for detection backbones before fine-tuning on COCO. Review the license on the official site before commercial use. Get it from objects365.org.

LVIS

LVIS reuses COCO's images but adds instance-level annotations for over 1,000 categories with a deliberately long-tailed frequency distribution, where many categories have only a handful of labeled examples. Use it when your evaluation needs to reflect real-world rarity — most objects a detector will encounter in deployment are not evenly distributed the way COCO's 80 balanced categories suggest. Get it from lvisdataset.org.

Autonomous driving

BDD100K

BDD100K, from UC Berkeley's DeepDrive project, is a large and diverse driving-video dataset covering 100,000 video clips captured across varied weather, time of day and driving conditions across the United States, with object detection, lane marking, drivable-area and instance segmentation annotations. Its diversity across geography and conditions makes it a stronger match for real-world driving perception research than COCO's static everyday photos. Review current license and access terms on the official site — registration is required. Get it from bdd-data.berkeley.edu.

nuScenes

nuScenes is a large-scale autonomous-driving dataset built around a full sensor suite — six cameras, five radars and one lidar with 360-degree coverage — capturing 1,000 driving scenes of 20 seconds each in Boston and Singapore, with 3D bounding boxes for 23 object classes across the full sensor set. Use it when your task requires multi-sensor 3D detection rather than 2D image-only detection, which none of the other datasets in this guide provide. Registration is required and licensing differs between academic and commercial use — confirm the current terms before a production deployment. Get it from nuscenes.org.

Satellite and aerial imagery

xView

xView, produced by the Defense Innovation Unit (DIUx) and the National Geospatial-Intelligence Agency (NGA), is one of the largest publicly available overhead-imagery detection datasets: about 1 million object instances across 60 classes, drawn from complex real-world scenes covering roughly 1,415 square kilometers at 0.3-meter resolution. It is the dataset to reach for when the task is detecting vehicles, buildings, or infrastructure from satellite or aerial imagery, a domain COCO, Open Images and the driving datasets above do not address. Review the terms and conditions on the official challenge site before use. Get it from xviewdataset.org.

Datasets side by side

Dataset	Domain	Scale	Notes
Open Images V7	General, broad category set	600 boxable classes, ~15.9M boxes	Check license per asset type
Objects365	General, balanced category set	365 classes, 2M images, 30M boxes	Check official license page
LVIS	General, long-tail	1,000+ categories on COCO images	Check official license page
BDD100K	Autonomous driving (2D video)	100K video clips, varied conditions	Registration required
nuScenes	Autonomous driving (3D, multi-sensor)	1,000 scenes, 23 3D classes	Academic / commercial terms differ
xView	Satellite / aerial imagery	~1M instances, 60 classes, 1,415 km²	Review challenge terms

How to choose

Start by naming what COCO cannot give you. If it is category breadth, choose between Open Images (largest vocabulary, uneven distribution) and Objects365 (smaller but more balanced vocabulary with more examples per class). If it is rare-category robustness, add LVIS rather than replacing COCO, since LVIS shares COCO's images. If your application is domain-specific — driving or aerial surveillance — general-object datasets will not transfer well regardless of their scale, so go straight to BDD100K or nuScenes for driving, or xView for aerial and satellite imagery. For any of these, confirm registration requirements and commercial-use terms early, since several require an account or a data-use agreement before download.

FAQ

Why would I need a dataset beyond COCO for object detection?

COCO covers only 80 everyday object categories on roughly 118k training images, which is not enough for tasks that need many more categories, rare long-tailed objects, domain-specific imagery like driving scenes or satellite photos, or simply more training volume than COCO provides. Open Images, Objects365 and LVIS extend category count and scale; BDD100K, nuScenes and xView cover domains COCO's everyday photos do not represent well.

Which dataset has the most object categories?

Open Images V7 has the largest label vocabulary, with over 20,000 image-level classes and 600 classes with bounding-box annotations across nearly 16 million boxes. LVIS focuses on a smaller but more evenly documented long-tailed set of over 1,000 categories built on COCO images. Objects365 sits in between with 365 categories but a larger, more balanced set of 2 million images and 30 million boxes.

Are autonomous-driving datasets like BDD100K and nuScenes free to use?

Both are free to download after registration, but check the specific license terms on each site before commercial use — some components may carry additional restrictions or require a separate commercial license for product use. Read the current terms on the official site rather than assuming free download means unrestricted commercial rights.

Still cannot find the right detection data? Leave it to us

If your target domain — medical imaging, retail shelf detection, industrial inspection — is not covered by these datasets, tell us your task goal and the conditions it must meet. DeepSData can run a real availability search and verify sources for you, checking reputable data platforms and judging matches and gaps against each requirement, with every source link kept traceable.

Open the dataset search assistant See research services →

New accounts get 3 free searches; every result comes with verifiable source links. See also curated datasets.