This page summarizes the Norwegian Collossal Corpus. The corpus is a collection of multiple smaller Norwegian corpuses that are suitable for training large language models. We have done extensive cleaning on the datasets, and have made them available in a common format.
A collection of documents and reports from the Norwegian government. The files are downloaded directly from the government API in a native xhtml-format that they use for creating both Word and pdf reports. In pre-processing typically headings, tables, footnotes etc are filtered out.
| License | Size | Words | Documents | Avg words per doc |
|---|
| NLOD 2.0 | 1.1 GB | 155,318,754 | 4,648 | 33,416 |
View dataset statistics
| Source | Words | Documents | Words/Document |
|---|
| government_nb | 136,708,062 | 3,557 | 38,433 |
| government_nn | 18,610,692 | 1,091 | 17,058 |
| Language | Words | Documents | Words/Document |
|---|
| no | 146,885,219 | 4,134 | 35,531 |
| nn | 8,246,002 | 500 | 16,492 |
| da | 182,720 | 13 | 14,055 |
| en | 4,813 | 1 | 4,813 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 155,318,754 | 4,648 | 33,416 |
NEED MORE TEXT HERE - WHO HAS COLLECTED THIS Also referred to as Stortingsforhandlingene. Collection of documents from the Norwegian Parliament. The corpus contains mainly OCRed reports. Part of the collection at the National Library.
| License | Size | Words | Documents | Avg words per doc |
|---|
| NLOD 2.0 | 8.0 GB | 1,301,766,124 | 9,528 | 136,625 |
View dataset statistics
| Source | Words | Documents | Words/Document |
|---|
| parliament | 1,301,766,124 | 9,528 | 136,625 |
| Language | Words | Documents | Words/Document |
|---|
| no | 885,007,390 | 6,478 | 136,617 |
| da | 416,758,734 | 3,050 | 136,642 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 1,301,766,124 | 9,528 | 136,625 |
NEED MORE TEXT HERE Also referred to as Evalueringsrapporter. The corpus contains mainly OCRed reports. Part of the collection at the National Library.
| License | Size | Words | Documents | Avg words per doc |
|---|
| NLOD 2.0 | 0.5 GB | 80,064,396 | 3,365 | 23,793 |
View dataset statistics
| Source | Words | Documents | Words/Document |
|---|
| publicreports | 80,064,396 | 3,365 | 23,793 |
| Language | Words | Documents | Words/Document |
|---|
| no | 65,026,196 | 2,721 | 23,897 |
| en | 13,093,711 | 556 | 23,549 |
| nn | 677,921 | 32 | 21,185 |
| sv | 381,007 | 14 | 27,214 |
| da | 290,817 | 12 | 24,234 |
| fr | 158,454 | 4 | 39,613 |
| es | 144,681 | 4 | 36,170 |
| et | 109,339 | 8 | 13,667 |
| zh | 88,848 | 3 | 29,616 |
| fi | 68,910 | 4 | 17,227 |
| gu | 16,739 | 1 | 16,739 |
| ru | 7,356 | 1 | 7,356 |
| uk | 268 | 1 | 268 |
| pt | 113 | 1 | 113 |
| it | 27 | 1 | 27 |
| de | 5 | 1 | 5 |
| ja | 4 | 1 | 4 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 80,064,396 | 3,365 | 23,793 |
CD/DVD last published in 2005. Contains a very complete collection of legal resources, including laws, NOUs etc. The collection is now out of database protextion. We have cleaned out resources that might contain personal information.
| License | Size | Words | Documents | Avg words per doc |
|---|
| NLOD 2.0 | 0.4 GB | 54,923,432 | 51,920 | 1,057 |
View dataset statistics
| Source | Words | Documents | Words/Document |
|---|
| lovdata_cd_odelsting_2005 | 37,295,277 | 1,987 | 18,769 |
| lovdata_cd_somb_rundskriv_2005 | 5,400,486 | 3,284 | 1,644 |
| lovdata_cd_sentrale_forskrifter_2005 | 5,178,057 | 11,745 | 440 |
| lovdata_cd_lokaleforskrifter_2005 | 2,233,543 | 22,824 | 97 |
| lovdata_cd_norgeslover_2005 | 1,802,578 | 1,419 | 1,270 |
| lovdata_cd_rtv_rundskriv_2005 | 1,392,547 | 9,831 | 141 |
| lovdata_cd_skatt_rundskriv_2005 | 1,138,339 | 411 | 2,769 |
| lovdata_cd_rundskriv_lovavdeling_2005 | 482,605 | 419 | 1,151 |
| Language | Words | Documents | Words/Document |
|---|
| no | 52,225,155 | 46,032 | 1,134 |
| nn | 2,369,924 | 3,261 | 726 |
| da | 173,296 | 1,146 | 151 |
| en | 104,233 | 423 | 246 |
| sv | 32,861 | 637 | 51 |
| es | 4,744 | 49 | 96 |
| nl | 3,818 | 212 | 18 |
| it | 3,768 | 6 | 628 |
| de | 3,179 | 58 | 54 |
| fr | 1,801 | 7 | 257 |
| pl | 513 | 57 | 9 |
| tr | 22 | 1 | 22 |
| fi | 14 | 4 | 3 |
| als | 12 | 3 | 4 |
| et | 11 | 2 | 5 |
| ca | 11 | 3 | 3 |
| zh | 11 | 3 | 3 |
| pt | 11 | 1 | 11 |
| la | 8 | 1 | 8 |
| eu | 7 | 2 | 3 |
| hr | 7 | 2 | 3 |
| ru | 5 | 2 | 2 |
| sk | 4 | 1 | 4 |
| lt | 3 | 1 | 3 |
| ro | 3 | 1 | 3 |
| sl | 3 | 1 | 3 |
| eo | 2 | 1 | 2 |
| am | 2 | 1 | 2 |
| cy | 2 | 1 | 2 |
| ms | 2 | 1 | 2 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 54,923,432 | 51,920 | 1,057 |
A collection of more than 9.2M public documents in pdf-format from 311 different sources. Collected from Språkbanken. We are using the part that is possible to read as text (roughly 6M documents). The largest sources are ssb (1.5M), regjeringen (1M), Uio (1M), fylkesmannen (0.7M), nve (0.4M), ntnu (0.3M), patentstyret (0.2M), vegvesenen (0.2M), fhi (0.2M).
| License | Size | Words | Documents | Avg words per doc |
|---|
| NLOD 2.0 | 14.0 GB | 1,905,481,776 | 6,735,367 | 282 |
View dataset statistics
| Source | Words | Documents | Words/Document |
|---|
| maalfrid_regjeringen | 368,581,046 | 940,936 | 391 |
| maalfrid_ssb | 286,024,711 | 871,262 | 328 |
| maalfrid_uio | 186,003,655 | 788,988 | 235 |
| maalfrid_fylkesmannen | 105,197,005 | 473,582 | 222 |
| maalfrid_nve | 68,194,530 | 308,924 | 220 |
| maalfrid_patentstyret | 66,482,941 | 218,930 | 303 |
| maalfrid_ntnu | 59,108,726 | 203,770 | 290 |
| maalfrid_vegvesen | 34,177,213 | 169,998 | 201 |
| maalfrid_fhi | 33,541,094 | 147,668 | 227 |
| maalfrid_norad | 33,454,341 | 95,191 | 351 |
| maalfrid_skatteetaten | 33,313,013 | 84,448 | 394 |
| maalfrid_uib | 29,049,820 | 118,328 | 245 |
| maalfrid_forskningsradet | 24,647,599 | 75,104 | 328 |
| maalfrid_nasjonalparkstyre | 21,795,981 | 95,990 | 227 |
| maalfrid_nmbu | 18,493,389 | 71,320 | 259 |
| maalfrid_oslomet | 18,140,360 | 48,140 | 376 |
| maalfrid_domstol | 17,073,430 | 52,233 | 326 |
| maalfrid_banenor | 16,805,767 | 71,933 | 233 |
| maalfrid_nav | 16,651,084 | 75,792 | 219 |
| maalfrid_landbruksdirektoratet | 13,398,273 | 49,021 | 273 |
| maalfrid_helsedirektoratet | 13,312,827 | 50,476 | 263 |
| maalfrid_nokut | 10,332,870 | 39,426 | 262 |
| maalfrid_hi | 10,272,572 | 39,923 | 257 |
| maalfrid_norges-bank | 10,135,291 | 37,988 | 266 |
| maalfrid_udir | 10,102,549 | 39,632 | 254 |
| maalfrid_vkm | 10,041,892 | 32,960 | 304 |
| maalfrid_nbim | 9,841,446 | 18,532 | 531 |
| maalfrid_miljodirektoratet | 9,704,586 | 35,482 | 273 |
| maalfrid_distriktssenteret | 9,598,021 | 39,415 | 243 |
| maalfrid_ngu | 9,454,229 | 35,414 | 266 |
| maalfrid_ptil | 9,416,592 | 35,024 | 268 |
| maalfrid_nord | 9,192,052 | 45,786 | 200 |
| maalfrid_fiskeridir | 8,482,774 | 34,167 | 248 |
| maalfrid_hivolda | 7,993,548 | 27,057 | 295 |
| maalfrid_difi | 7,971,205 | 36,605 | 217 |
| maalfrid_mattilsynet | 7,663,012 | 27,614 | 277 |
| maalfrid_havarikommisjonen | 7,607,533 | 25,552 | 297 |
| maalfrid_kulturradet | 7,364,353 | 22,951 | 320 |
| maalfrid_ks | 7,065,314 | 28,029 | 252 |
| maalfrid_kystverket | 6,870,772 | 31,694 | 216 |
| maalfrid_udi | 6,566,701 | 19,529 | 336 |
| maalfrid_uia | 6,094,660 | 24,397 | 249 |
| maalfrid_hjelpemiddeldatabasen | 6,029,920 | 34,946 | 172 |
| maalfrid_khrono | 5,993,140 | 20,431 | 293 |
| maalfrid_helsetilsynet | 5,913,602 | 18,721 | 315 |
| maalfrid_moreforsk | 5,755,447 | 22,089 | 260 |
| maalfrid_jernbanedirektoratet | 5,589,074 | 22,150 | 252 |
| maalfrid_veiviseren | 5,438,468 | 18,441 | 294 |
| maalfrid_dsb | 5,312,625 | 18,200 | 291 |
| maalfrid_husbanken | 4,810,203 | 15,375 | 312 |
| maalfrid_legemiddelverket | 4,795,154 | 20,634 | 232 |
| maalfrid_vetinst | 4,776,782 | 14,839 | 321 |
| maalfrid_imdi | 4,744,408 | 15,642 | 303 |
| maalfrid_forsvarsbygg | 4,672,409 | 19,287 | 242 |
| maalfrid_sdir | 4,640,185 | 15,547 | 298 |
| maalfrid_konkurransetilsynet | 4,618,588 | 12,912 | 357 |
| maalfrid_arkivverket | 4,603,524 | 16,899 | 272 |
| maalfrid_dsa | 4,595,530 | 16,242 | 282 |
| maalfrid_hiof | 4,580,991 | 23,675 | 193 |
| maalfrid_ehelse | 4,478,908 | 23,074 | 194 |
| maalfrid_inn | 4,420,070 | 26,840 | 164 |
| maalfrid_klagenemndssekretariatet | 4,287,067 | 12,208 | 351 |
| maalfrid_sprakradet | 4,180,404 | 15,521 | 269 |
| maalfrid_nhh | 4,063,920 | 16,068 | 252 |
| maalfrid_dibk | 4,058,208 | 15,855 | 255 |
| maalfrid_kartverket | 3,814,376 | 19,110 | 199 |
| maalfrid_riksrevisjonen | 3,783,728 | 11,216 | 337 |
| maalfrid_toll | 3,595,842 | 14,122 | 254 |
| maalfrid_nibio | 3,531,231 | 17,464 | 202 |
| maalfrid_met | 3,528,846 | 18,689 | 188 |
| maalfrid_bufdir | 3,425,362 | 11,720 | 292 |
| maalfrid_artsdatabanken | 3,275,378 | 9,242 | 354 |
| maalfrid_politiet | 3,239,913 | 10,728 | 302 |
| maalfrid_nkom | 3,197,196 | 10,214 | 313 |
| maalfrid_vestlandfylke | 3,127,665 | 12,337 | 253 |
| maalfrid_uis | 2,988,424 | 10,045 | 297 |
| maalfrid_sykkelbynettverket | 2,880,916 | 12,086 | 238 |
| maalfrid_nlr | 2,702,753 | 16,178 | 167 |
| maalfrid_seniorporten | 2,672,667 | 8,295 | 322 |
| maalfrid_npd | 2,657,179 | 10,989 | 241 |
| maalfrid_custompublish | 2,493,062 | 9,404 | 265 |
| maalfrid_aldringoghelse | 2,475,601 | 6,927 | 357 |
| maalfrid_bioteknologiradet | 2,450,272 | 6,135 | 399 |
| maalfrid_nyemetoder | 2,426,982 | 10,999 | 220 |
| maalfrid_arbeidstilsynet | 2,426,255 | 7,030 | 345 |
| maalfrid_riksantikvaren | 2,300,159 | 8,933 | 257 |
| maalfrid_sjt | 2,292,578 | 11,455 | 200 |
| maalfrid_hvl | 2,194,063 | 9,604 | 228 |
| maalfrid_luftfartstilsynet | 2,149,215 | 10,092 | 212 |
| maalfrid_dfo | 2,123,792 | 9,383 | 226 |
| maalfrid_ldo | 2,093,301 | 7,471 | 280 |
| maalfrid_kompetansenorge | 1,997,361 | 10,496 | 190 |
| maalfrid_forbrukerradet | 1,992,302 | 7,493 | 265 |
| maalfrid_himolde | 1,959,626 | 10,200 | 192 |
| maalfrid_usn | 1,828,928 | 7,553 | 242 |
| maalfrid_naku | 1,786,086 | 5,328 | 335 |
| maalfrid_medietilsynet | 1,648,462 | 6,758 | 243 |
| maalfrid_matematikksenteret | 1,608,332 | 7,474 | 215 |
| maalfrid_diku | 1,579,996 | 6,383 | 247 |
| maalfrid_forskningsetikk | 1,573,014 | 5,653 | 278 |
| maalfrid_godeidrettsanlegg | 1,539,910 | 6,252 | 246 |
| maalfrid_dirmin | 1,500,122 | 5,427 | 276 |
| maalfrid_diskrimineringsnemnda | 1,498,443 | 4,270 | 350 |
| maalfrid_naturfag | 1,481,316 | 6,108 | 242 |
| maalfrid_arbeidsretten | 1,473,299 | 4,864 | 302 |
| maalfrid_fellesstudentsystem | 1,392,117 | 10,553 | 131 |
| maalfrid_nupi | 1,322,743 | 5,628 | 235 |
| maalfrid_kriminalitetsforebygging | 1,223,976 | 4,769 | 256 |
| maalfrid_anskaffelser | 1,214,995 | 5,602 | 216 |
| maalfrid_folketrygdfondet | 1,212,747 | 4,347 | 278 |
| maalfrid_miljopakken | 1,195,869 | 5,634 | 212 |
| maalfrid_nih | 1,146,471 | 5,415 | 211 |
| maalfrid_statsbygg | 1,125,666 | 4,520 | 249 |
| maalfrid_nb | 1,085,457 | 4,251 | 255 |
| maalfrid_unit | 1,072,199 | 6,476 | 165 |
| maalfrid_npolar | 1,071,381 | 2,708 | 395 |
| maalfrid_valgdirektoratet | 1,028,381 | 9,316 | 110 |
| maalfrid_barneombudet | 1,001,224 | 2,863 | 349 |
| maalfrid_datatilsynet | 990,582 | 3,018 | 328 |
| maalfrid_lottstift | 985,351 | 3,676 | 268 |
| maalfrid_aho | 977,116 | 4,637 | 210 |
| maalfrid_sykehuspartner | 961,362 | 4,693 | 204 |
| maalfrid_naturfagsenteret | 922,174 | 3,957 | 233 |
| maalfrid_khio | 869,917 | 3,457 | 251 |
| maalfrid_spesialenheten | 845,989 | 2,188 | 386 |
| maalfrid_xn--miljlftet-o8ab | 822,783 | 3,468 | 237 |
| maalfrid_samordnaopptak | 805,644 | 2,415 | 333 |
| maalfrid_helsenorge | 802,003 | 3,116 | 257 |
| maalfrid_skrivesenteret | 793,053 | 4,250 | 186 |
| maalfrid_mareano | 784,843 | 3,821 | 205 |
| maalfrid_fiskeridirektoratet | 772,720 | 2,499 | 309 |
| maalfrid_sykehusinnkjop | 754,616 | 4,440 | 169 |
| maalfrid_matportalen | 641,663 | 2,413 | 265 |
| maalfrid_spk | 621,687 | 2,181 | 285 |
| maalfrid_pasientsikkerhetsprogrammet | 610,855 | 4,796 | 127 |
| maalfrid_justervesenet | 607,767 | 1,946 | 312 |
| maalfrid_nhn | 594,591 | 3,665 | 162 |
| maalfrid_sshf | 589,448 | 1,950 | 302 |
| maalfrid_bibliotekutvikling | 573,724 | 3,295 | 174 |
| maalfrid_nysgjerrigper | 572,860 | 3,088 | 185 |
| maalfrid_nodnett | 549,483 | 2,743 | 200 |
| maalfrid_giek | 525,782 | 1,840 | 285 |
| maalfrid_une | 524,664 | 1,281 | 409 |
| maalfrid_samas | 512,469 | 2,610 | 196 |
| maalfrid_kriminalomsorgen | 506,869 | 1,990 | 254 |
| maalfrid_kjonnsforskning | 495,072 | 1,464 | 338 |
| maalfrid_kunstkultursenteret | 478,748 | 1,460 | 327 |
| maalfrid_nynorsksenteret | 472,484 | 2,147 | 220 |
| maalfrid_ceres | 457,001 | 1,988 | 229 |
| maalfrid_stami | 456,707 | 1,190 | 383 |
| maalfrid_nsm | 452,597 | 1,573 | 287 |
| maalfrid_gjenopptakelse | 430,682 | 1,501 | 286 |
| maalfrid_nfi | 430,469 | 1,557 | 276 |
| maalfrid_nidsenter | 416,329 | 1,657 | 251 |
| maalfrid_nasjonalmuseet | 396,739 | 1,106 | 358 |
| maalfrid_forbrukertilsynet | 395,317 | 1,252 | 315 |
| maalfrid_natursekken | 389,147 | 3,657 | 106 |
| maalfrid_fordelingsutvalget | 362,923 | 1,416 | 256 |
| maalfrid_digdir | 358,558 | 2,159 | 166 |
| maalfrid_forsvaret | 339,218 | 1,243 | 272 |
| maalfrid_beccle | 337,729 | 1,554 | 217 |
| maalfrid_romsenter | 335,107 | 1,154 | 290 |
| maalfrid_geonorge | 306,914 | 1,658 | 185 |
| maalfrid_universell | 269,369 | 2,206 | 122 |
| maalfrid_ovf | 267,941 | 950 | 282 |
| maalfrid_forbrukereuropa | 264,366 | 1,043 | 253 |
| maalfrid_politihogskolen | 264,192 | 1,253 | 210 |
| maalfrid_vinmonopolet | 250,998 | 689 | 364 |
| maalfrid_energimerking | 243,288 | 1,061 | 229 |
| maalfrid_ombudsmann | 235,226 | 432 | 544 |
| maalfrid_vea-fs | 231,109 | 1,293 | 178 |
| maalfrid_traumebevisst | 228,320 | 2,482 | 91 |
| maalfrid_npe | 208,768 | 1,018 | 205 |
| maalfrid_pkh | 206,925 | 814 | 254 |
| maalfrid_opplaringslovutvalget | 198,545 | 561 | 353 |
| maalfrid_helfo | 197,334 | 1,005 | 196 |
| maalfrid_regionaleforskningsfond | 191,416 | 1,010 | 189 |
| maalfrid_nafkam | 180,622 | 582 | 310 |
| maalfrid_jernbanemagasinet | 178,723 | 422 | 423 |
| maalfrid_polarhistorie | 176,126 | 393 | 448 |
| maalfrid_aasentunet | 165,549 | 543 | 304 |
| maalfrid_riksteatret | 161,970 | 809 | 200 |
| maalfrid_realfagsloyper | 160,310 | 765 | 209 |
| maalfrid_koro | 156,518 | 584 | 268 |
| maalfrid_squarespace | 149,259 | 515 | 289 |
| maalfrid_politietssikkerhetstjeneste | 146,346 | 478 | 306 |
| maalfrid_unknown | 142,298 | 715 | 199 |
| maalfrid_whocc | 122,839 | 664 | 184 |
| maalfrid_konfliktraadet | 122,030 | 379 | 321 |
| maalfrid_okokrim | 119,794 | 381 | 314 |
| maalfrid_brreg | 115,114 | 583 | 197 |
| maalfrid_riksmekleren | 113,287 | 570 | 198 |
| maalfrid_sismo | 112,976 | 316 | 357 |
| maalfrid_akkreditert | 101,275 | 513 | 197 |
| maalfrid_sivilforsvaret | 101,178 | 528 | 191 |
| maalfrid_radetfordyreetikk | 100,021 | 446 | 224 |
| maalfrid_lanekassen | 97,196 | 309 | 314 |
| maalfrid_digidel | 96,967 | 621 | 156 |
| maalfrid_uit | 92,451 | 612 | 151 |
| maalfrid_generaladvokaten | 91,998 | 297 | 309 |
| maalfrid_nyinorge | 90,938 | 204 | 445 |
| maalfrid_lokforerskolen | 90,255 | 478 | 188 |
| maalfrid_varsom | 87,050 | 576 | 151 |
| maalfrid_ffi | 82,147 | 224 | 366 |
| maalfrid_kulturminnefondet | 81,683 | 424 | 192 |
| maalfrid_unesco | 78,677 | 388 | 202 |
| maalfrid_yrkesfisker | 76,760 | 512 | 149 |
| maalfrid_dekom | 74,066 | 1,331 | 55 |
| maalfrid_omsorgsforskning | 73,528 | 332 | 221 |
| maalfrid_lektor2 | 70,477 | 561 | 125 |
| maalfrid_openaccess | 65,385 | 197 | 331 |
| maalfrid_ssn | 64,111 | 308 | 208 |
| maalfrid_lokalhistorie | 61,885 | 250 | 247 |
| maalfrid_laudim | 59,669 | 402 | 148 |
| maalfrid_nlb | 58,927 | 206 | 286 |
| maalfrid_riksadvokaten | 57,938 | 156 | 371 |
| maalfrid_denkulturelleskolesekken | 46,768 | 248 | 188 |
| maalfrid_sivilrett | 45,214 | 145 | 311 |
| maalfrid_htu | 43,778 | 171 | 256 |
| maalfrid_yr | 41,565 | 575 | 72 |
| maalfrid_informasjonskompetanse | 40,989 | 334 | 122 |
| maalfrid_finansportalen | 40,333 | 187 | 215 |
| maalfrid_dep | 38,882 | 126 | 308 |
| maalfrid_kulturped | 37,718 | 99 | 380 |
| maalfrid_feide | 37,583 | 274 | 137 |
| maalfrid_fug | 35,253 | 123 | 286 |
| maalfrid_kulturoghelse | 34,762 | 189 | 183 |
| maalfrid_helseklage | 33,612 | 127 | 264 |
| maalfrid_nbsk | 31,334 | 215 | 145 |
| maalfrid_matogindustri | 31,232 | 207 | 150 |
| maalfrid_sinn | 28,114 | 154 | 182 |
| maalfrid_transport21 | 25,691 | 91 | 282 |
| maalfrid_vergemal | 24,189 | 80 | 302 |
| maalfrid_konkursradet | 24,072 | 78 | 308 |
| maalfrid_norec | 22,496 | 78 | 288 |
| maalfrid_pts | 21,346 | 81 | 263 |
| maalfrid_nasjonaleturistveger | 20,237 | 111 | 182 |
| maalfrid_hjelpelinjen | 19,476 | 86 | 226 |
| maalfrid_iearth | 19,418 | 150 | 129 |
| maalfrid_russamtalen | 19,035 | 69 | 275 |
| maalfrid_xn--kvinneligomskjring-1ub | 18,607 | 79 | 235 |
| maalfrid_memu | 17,820 | 101 | 176 |
| maalfrid_nynorskbok | 17,769 | 98 | 181 |
| maalfrid_regjeringsadvokaten | 17,416 | 55 | 316 |
| maalfrid_xn--forskerfr-t8a | 16,827 | 177 | 95 |
| maalfrid_xn--tilbakefring-2jb | 15,814 | 49 | 322 |
| maalfrid_ringerikefengsel | 15,669 | 28 | 559 |
| maalfrid_skattefunn | 15,625 | 54 | 289 |
| maalfrid_skeivtarkiv | 15,537 | 69 | 225 |
| maalfrid_fordelingsutvalet | 15,473 | 35 | 442 |
| maalfrid_samfunnskunnskap | 15,110 | 60 | 251 |
| maalfrid_shiprep | 14,632 | 146 | 100 |
| maalfrid_sevuppt | 14,087 | 55 | 256 |
| maalfrid_haldenfengsel | 13,655 | 38 | 359 |
| maalfrid_forbrukerklageutvalget | 13,472 | 51 | 264 |
| maalfrid_mhfa | 12,591 | 146 | 86 |
| maalfrid_ah | 11,787 | 36 | 327 |
| maalfrid_nettvett | 11,353 | 44 | 258 |
| maalfrid_uh-it | 11,158 | 281 | 39 |
| maalfrid_fishgen | 10,318 | 29 | 355 |
| maalfrid_designavgang | 10,164 | 75 | 135 |
| maalfrid_global | 9,363 | 43 | 217 |
| maalfrid_valg | 8,797 | 48 | 183 |
| maalfrid_havmiljo | 8,734 | 69 | 126 |
| maalfrid_altinn | 7,945 | 50 | 158 |
| maalfrid_miljoklagenemnda | 7,797 | 35 | 222 |
| maalfrid_spinn-inn | 7,699 | 48 | 160 |
| maalfrid_kantinekurset | 7,397 | 54 | 136 |
| maalfrid_bastoyfengsel | 7,142 | 56 | 127 |
| maalfrid_norskpetroleum | 6,107 | 120 | 50 |
| maalfrid_voldsoffererstatning | 6,079 | 27 | 225 |
| maalfrid_musikkbasertmiljobehandling | 5,186 | 39 | 132 |
| maalfrid_prosjektveiviseren | 5,154 | 15 | 343 |
| maalfrid_aldersvennlig | 4,919 | 32 | 153 |
| maalfrid_barentswatch | 4,829 | 32 | 150 |
| maalfrid_fmfiavo@fylkesmannen | 4,740 | 69 | 68 |
| maalfrid_kk-utvalget | 4,697 | 19 | 247 |
| maalfrid_agropub | 4,434 | 17 | 260 |
| maalfrid_utdanningiverden | 4,369 | 14 | 312 |
| maalfrid_overgangsbolig | 3,862 | 36 | 107 |
| maalfrid_forsvaretsmuseer | 3,840 | 35 | 109 |
| maalfrid_okopark | 3,282 | 12 | 273 |
| maalfrid_pst | 2,866 | 14 | 204 |
| maalfrid_sikkerhverdag | 2,786 | 19 | 146 |
| maalfrid_arkitektur | 2,436 | 15 | 162 |
| maalfrid_velgekte | 2,287 | 10 | 228 |
| maalfrid_addlab | 2,109 | 12 | 175 |
| maalfrid_romerikefengsel | 2,088 | 19 | 109 |
| maalfrid_utdanning | 2,009 | 12 | 167 |
| maalfrid_grunderskolen | 1,994 | 7 | 284 |
| maalfrid_umb | 1,958 | 9 | 217 |
| maalfrid_oslofengsel | 1,756 | 8 | 219 |
| maalfrid_hjorteviltregisteret | 1,600 | 5 | 320 |
| maalfrid_alleteller | 1,511 | 7 | 215 |
| maalfrid_webhuset | 1,409 | 5 | 281 |
| maalfrid_lykillinn | 1,349 | 4 | 337 |
| maalfrid_kulturfag | 1,215 | 6 | 202 |
| maalfrid_unimus | 940 | 4 | 235 |
| maalfrid_anleggsregisteret | 928 | 5 | 185 |
| maalfrid_mangfoldsprisen | 597 | 3 | 199 |
| maalfrid_algae2future | 456 | 8 | 57 |
| maalfrid_mammapresenterer | 447 | 2 | 223 |
| maalfrid_karriereveiledning | 391 | 27 | 14 |
| maalfrid_nodsms | 351 | 4 | 87 |
| maalfrid_kildekompasset | 302 | 1 | 302 |
| maalfrid_praksisfou | 297 | 1 | 297 |
| maalfrid_retttilaalese | 246 | 3 | 82 |
| maalfrid_indreostfoldfengsel | 215 | 3 | 71 |
| maalfrid_xn--kroppsvingsforskning-gcc | 205 | 2 | 102 |
| maalfrid_pahoyden | 154 | 1 | 154 |
| maalfrid_norren | 42 | 1 | 42 |
| Language | Words | Documents | Words/Document |
|---|
| no | 1,248,901,257 | 4,462,909 | 279 |
| en | 422,678,950 | 1,456,480 | 290 |
| da | 73,779,920 | 256,659 | 287 |
| nn | 63,661,272 | 206,735 | 307 |
| fr | 45,122,943 | 107,315 | 420 |
| de | 11,584,819 | 60,772 | 190 |
| sv | 10,007,099 | 50,064 | 199 |
| es | 8,138,812 | 31,031 | 262 |
| pt | 2,483,288 | 14,797 | 167 |
| fi | 2,406,210 | 10,484 | 229 |
| oc | 2,170,769 | 4,988 | 435 |
| nl | 1,562,699 | 6,789 | 230 |
| uk | 1,446,456 | 4,296 | 336 |
| zh | 1,406,637 | 7,759 | 181 |
| ca | 1,388,067 | 3,630 | 382 |
| ru | 1,205,439 | 5,669 | 212 |
| it | 950,452 | 6,689 | 142 |
| et | 938,286 | 3,968 | 236 |
| cs | 887,973 | 4,301 | 206 |
| eu | 851,535 | 3,160 | 269 |
| pl | 735,829 | 4,909 | 149 |
| fa | 503,664 | 2,060 | 244 |
| ja | 349,742 | 3,515 | 99 |
| is | 309,524 | 995 | 311 |
| id | 223,364 | 1,255 | 177 |
| ar | 212,455 | 1,181 | 179 |
| so | 131,743 | 605 | 217 |
| el | 120,377 | 617 | 195 |
| hu | 119,131 | 1,277 | 93 |
| vi | 96,710 | 497 | 194 |
| sl | 93,656 | 649 | 144 |
| tr | 91,012 | 1,010 | 90 |
| ro | 76,959 | 427 | 180 |
| lt | 67,317 | 552 | 121 |
| sr | 65,928 | 791 | 83 |
| gl | 65,517 | 579 | 113 |
| la | 62,069 | 461 | 134 |
| th | 53,793 | 359 | 149 |
| hr | 48,627 | 472 | 103 |
| am | 46,475 | 326 | 142 |
| ko | 44,264 | 884 | 50 |
| ml | 35,492 | 151 | 235 |
| sq | 31,182 | 148 | 210 |
| tl | 30,964 | 163 | 189 |
| kk | 27,686 | 69 | 401 |
| mn | 21,540 | 22 | 979 |
| sw | 18,626 | 64 | 291 |
| pnb | 18,502 | 81 | 228 |
| eo | 17,760 | 440 | 40 |
| sk | 17,616 | 195 | 90 |
| bg | 16,521 | 97 | 170 |
| ur | 15,797 | 140 | 112 |
| sh | 14,138 | 120 | 117 |
| mk | 13,479 | 65 | 207 |
| lv | 9,389 | 60 | 156 |
| ckb | 9,350 | 44 | 212 |
| ku | 9,058 | 54 | 167 |
| ast | 7,910 | 63 | 125 |
| uz | 6,883 | 35 | 196 |
| az | 6,757 | 44 | 153 |
| ceb | 5,492 | 227 | 24 |
| war | 4,149 | 95 | 43 |
| ta | 3,990 | 58 | 68 |
| ms | 3,655 | 94 | 38 |
| hy | 3,279 | 30 | 109 |
| fy | 2,837 | 22 | 128 |
| hi | 2,810 | 40 | 70 |
| ht | 2,534 | 11 | 230 |
| jv | 2,042 | 31 | 65 |
| cy | 1,988 | 40 | 49 |
| su | 1,840 | 23 | 80 |
| ps | 1,832 | 15 | 122 |
| als | 1,805 | 35 | 51 |
| af | 1,703 | 20 | 85 |
| bs | 1,524 | 24 | 63 |
| qu | 1,484 | 13 | 114 |
| nds | 1,328 | 75 | 17 |
| be | 1,209 | 33 | 36 |
| my | 1,115 | 16 | 69 |
| ga | 955 | 25 | 38 |
| mt | 940 | 12 | 78 |
| si | 898 | 22 | 40 |
| te | 853 | 17 | 50 |
| ilo | 801 | 16 | 50 |
| io | 689 | 10 | 68 |
| tt | 675 | 20 | 33 |
| km | 647 | 11 | 58 |
| jbo | 621 | 27 | 23 |
| gn | 595 | 7 | 85 |
| ug | 581 | 6 | 96 |
| kv | 562 | 3 | 187 |
| br | 533 | 20 | 26 |
| kn | 531 | 19 | 27 |
| bn | 490 | 20 | 24 |
| pam | 476 | 1 | 476 |
| pa | 413 | 15 | 27 |
| he | 396 | 14 | 28 |
| ka | 326 | 17 | 19 |
| kw | 324 | 4 | 81 |
| vep | 296 | 11 | 26 |
| yo | 261 | 5 | 52 |
| gu | 234 | 12 | 19 |
| ky | 232 | 11 | 21 |
| azb | 216 | 1 | 216 |
| ba | 203 | 5 | 40 |
| gom | 164 | 9 | 18 |
| wa | 130 | 18 | 7 |
| tg | 129 | 3 | 43 |
| ia | 125 | 11 | 11 |
| mr | 122 | 6 | 20 |
| lmo | 87 | 23 | 3 |
| lb | 68 | 15 | 4 |
| vec | 67 | 3 | 22 |
| rue | 67 | 2 | 33 |
| pms | 63 | 8 | 7 |
| min | 60 | 6 | 10 |
| ne | 51 | 5 | 10 |
| hsb | 51 | 2 | 25 |
| cbk | 46 | 2 | 23 |
| or | 44 | 2 | 22 |
| ie | 38 | 5 | 7 |
| tk | 36 | 4 | 9 |
| eml | 31 | 4 | 7 |
| arz | 31 | 1 | 31 |
| sco | 30 | 1 | 30 |
| gd | 29 | 2 | 14 |
| bar | 27 | 2 | 13 |
| li | 22 | 3 | 7 |
| diq | 20 | 2 | 10 |
| yue | 19 | 1 | 19 |
| dsb | 19 | 1 | 19 |
| as | 17 | 1 | 17 |
| os | 15 | 2 | 7 |
| wuu | 14 | 1 | 14 |
| mg | 14 | 2 | 7 |
| sd | 14 | 1 | 14 |
| nah | 14 | 2 | 7 |
| cv | 12 | 1 | 12 |
| scn | 9 | 2 | 4 |
| bh | 8 | 1 | 8 |
| bcl | 8 | 1 | 8 |
| ce | 4 | 1 | 4 |
| new | 4 | 1 | 4 |
| frr | 3 | 1 | 3 |
| vo | 3 | 2 | 1 |
| gv | 3 | 1 | 3 |
| mzn | 3 | 1 | 3 |
| lo | 2 | 1 | 2 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 1,905,481,776 | 6,735,367 | 282 |
A large collection of out-of-copyright newspapers from the National Library of Norway. This collection both has OCR scanned newspapers, and newspapers that are delivered to the library as pdf and where we are able to extract the text directly.
| License | Size | Words | Documents | Avg words per doc |
|---|
| CC0 1.0 | 14.0 GB | 2,019,172,625 | 10,096,424 | 199 |
View dataset statistics
| Source | Words | Documents | Words/Document |
|---|
| newspaper_ocr | 2,019,172,625 | 10,096,424 | 199 |
| Language | Words | Documents | Words/Document |
|---|
| no | 1,854,790,815 | 9,011,769 | 205 |
| nn | 126,257,684 | 526,902 | 239 |
| da | 14,101,047 | 254,996 | 55 |
| en | 9,187,689 | 93,497 | 98 |
| fi | 4,264,629 | 37,132 | 114 |
| sv | 3,011,755 | 63,999 | 47 |
| et | 2,326,165 | 20,753 | 112 |
| cs | 1,781,741 | 17,925 | 99 |
| es | 1,639,849 | 10,848 | 151 |
| de | 679,620 | 23,312 | 29 |
| hu | 480,299 | 7,360 | 65 |
| fr | 128,379 | 2,517 | 51 |
| nl | 103,567 | 4,714 | 21 |
| pl | 103,427 | 4,420 | 23 |
| sl | 48,603 | 1,224 | 39 |
| pt | 46,407 | 1,772 | 26 |
| it | 33,927 | 1,646 | 20 |
| ru | 29,604 | 1,029 | 28 |
| sk | 22,246 | 814 | 27 |
| uk | 21,106 | 792 | 26 |
| ca | 17,842 | 912 | 19 |
| hr | 14,090 | 586 | 24 |
| ja | 10,886 | 1,519 | 7 |
| id | 9,275 | 809 | 11 |
| sr | 7,805 | 198 | 39 |
| zh | 7,101 | 548 | 12 |
| is | 5,428 | 234 | 23 |
| vi | 4,631 | 376 | 12 |
| tr | 3,703 | 634 | 5 |
| ms | 3,528 | 395 | 8 |
| sq | 3,088 | 88 | 35 |
| lt | 3,074 | 316 | 9 |
| ga | 3,024 | 157 | 19 |
| oc | 2,784 | 42 | 66 |
| sh | 2,259 | 93 | 24 |
| eu | 2,229 | 106 | 21 |
| eo | 2,002 | 389 | 5 |
| la | 1,799 | 296 | 6 |
| ro | 1,447 | 192 | 7 |
| gl | 798 | 119 | 6 |
| bs | 770 | 43 | 17 |
| ko | 766 | 111 | 6 |
| lv | 488 | 61 | 8 |
| war | 443 | 40 | 11 |
| nds | 437 | 83 | 5 |
| ceb | 378 | 61 | 6 |
| ilo | 325 | 9 | 36 |
| jv | 297 | 59 | 5 |
| fy | 277 | 43 | 6 |
| ur | 243 | 37 | 6 |
| ml | 190 | 10 | 19 |
| br | 172 | 22 | 7 |
| fa | 160 | 29 | 5 |
| ar | 160 | 21 | 7 |
| kw | 148 | 14 | 10 |
| be | 145 | 20 | 7 |
| als | 137 | 29 | 4 |
| el | 120 | 15 | 8 |
| vep | 117 | 21 | 5 |
| uz | 107 | 24 | 4 |
| cy | 86 | 13 | 6 |
| hy | 73 | 11 | 6 |
| ast | 73 | 12 | 6 |
| gn | 70 | 5 | 14 |
| th | 63 | 14 | 4 |
| su | 62 | 7 | 8 |
| min | 60 | 12 | 5 |
| tl | 57 | 14 | 4 |
| af | 53 | 14 | 3 |
| si | 53 | 8 | 6 |
| lmo | 47 | 4 | 11 |
| pms | 45 | 7 | 6 |
| sw | 44 | 8 | 5 |
| az | 42 | 11 | 3 |
| lb | 38 | 9 | 4 |
| te | 35 | 1 | 35 |
| bg | 34 | 5 | 6 |
| mt | 34 | 4 | 8 |
| ka | 33 | 4 | 8 |
| sco | 31 | 5 | 6 |
| io | 22 | 4 | 5 |
| ie | 21 | 6 | 3 |
| os | 21 | 1 | 21 |
| yo | 20 | 4 | 5 |
| mg | 16 | 4 | 4 |
| bar | 16 | 4 | 4 |
| he | 16 | 4 | 4 |
| nap | 16 | 1 | 16 |
| mr | 16 | 4 | 4 |
| pa | 16 | 1 | 16 |
| ia | 15 | 4 | 3 |
| wa | 14 | 2 | 7 |
| qu | 14 | 1 | 14 |
| ku | 13 | 3 | 4 |
| kn | 13 | 3 | 4 |
| rm | 12 | 2 | 6 |
| frr | 10 | 2 | 5 |
| gom | 10 | 3 | 3 |
| bn | 9 | 2 | 4 |
| tt | 9 | 2 | 4 |
| kk | 8 | 2 | 4 |
| li | 7 | 3 | 2 |
| eml | 7 | 2 | 3 |
| mk | 7 | 2 | 3 |
| co | 7 | 1 | 7 |
| hsb | 6 | 1 | 6 |
| jbo | 5 | 2 | 2 |
| ne | 5 | 1 | 5 |
| pam | 4 | 1 | 4 |
| ta | 3 | 1 | 3 |
| hi | 2 | 1 | 2 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 48,107 | 321 | 149 |
| 2010 | 345,461,597 | 2,469,655 | 1,414 |
| 2000 | 399,916,520 | 1,741,144 | 2,334 |
| 1990 | 668,123,800 | 2,563,932 | 2,589 |
| 1980 | 124,185,906 | 549,411 | 2,267 |
| 1970 | 168,128,159 | 847,524 | 1,973 |
| 1960 | 134,790,494 | 852,668 | 1,581 |
| 1950 | 82,534,163 | 489,139 | 1,677 |
| 1940 | 95,983,879 | 582,630 | 1,609 |
A Norwegian Newspaper Corpus with texts from online bokmål newspapers between 1998 and 2019 collected by Språkbanken.
| License | Size | Words | Documents | Avg words per doc |
|---|
| CC BY-NC 2.0 | 3.7 GB | 541,481,947 | 3,695,943 | 146 |
View dataset statistics
| Source | Words | Documents | Words/Document |
|---|
| newspapers_online_nb | 498,332,371 | 3,524,770 | 141 |
| newspapers_online_nn | 43,149,576 | 171,173 | 252 |
| Language | Words | Documents | Words/Document |
|---|
| no | 501,126,127 | 3,491,685 | 143 |
| nn | 36,177,754 | 134,633 | 268 |
| da | 3,686,792 | 55,966 | 65 |
| en | 236,278 | 5,749 | 41 |
| sv | 146,978 | 4,343 | 33 |
| es | 57,860 | 954 | 60 |
| de | 22,640 | 1,175 | 19 |
| nl | 8,459 | 285 | 29 |
| ru | 3,852 | 74 | 52 |
| it | 3,397 | 219 | 15 |
| fr | 2,639 | 185 | 14 |
| pl | 2,064 | 132 | 15 |
| cs | 1,395 | 60 | 23 |
| pt | 1,220 | 74 | 16 |
| fi | 1,016 | 30 | 33 |
| sh | 707 | 4 | 176 |
| uk | 476 | 27 | 17 |
| ja | 372 | 65 | 5 |
| hr | 216 | 19 | 11 |
| la | 207 | 13 | 15 |
| tr | 188 | 20 | 9 |
| hu | 181 | 23 | 7 |
| ca | 174 | 35 | 4 |
| sl | 158 | 9 | 17 |
| id | 142 | 20 | 7 |
| et | 88 | 20 | 4 |
| ro | 87 | 13 | 6 |
| wa | 80 | 18 | 4 |
| eo | 67 | 14 | 4 |
| eu | 66 | 12 | 5 |
| vi | 59 | 11 | 5 |
| zh | 35 | 10 | 3 |
| is | 26 | 4 | 6 |
| lt | 16 | 4 | 4 |
| ms | 14 | 5 | 2 |
| ko | 13 | 3 | 4 |
| nds | 10 | 4 | 2 |
| lb | 9 | 2 | 4 |
| mg | 8 | 2 | 4 |
| als | 7 | 1 | 7 |
| ceb | 7 | 3 | 2 |
| vep | 6 | 2 | 3 |
| war | 6 | 2 | 3 |
| ia | 6 | 1 | 6 |
| mt | 6 | 1 | 6 |
| pms | 5 | 1 | 5 |
| be | 4 | 1 | 4 |
| io | 4 | 1 | 4 |
| sr | 4 | 1 | 4 |
| fa | 4 | 1 | 4 |
| gl | 4 | 1 | 4 |
| kw | 3 | 1 | 3 |
| sk | 3 | 1 | 3 |
| bar | 3 | 1 | 3 |
| jv | 2 | 1 | 2 |
| ga | 2 | 1 | 2 |
| ar | 1 | 1 | 1 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 541,481,947 | 3,695,943 | 146 |
A large collection of out-of-copyright books from the National Library of Norway. This is OCR scanned documents only.
| License | Size | Words | Documents | Avg words per doc |
|---|
| CC0 1.0 | 6.2 GB | 861,465,907 | 24,253 | 35,519 |
View dataset statistics
| Language | Words | Documents | Words/Document |
|---|
| da | 451,088,972 | 8,443 | 53,427 |
| no | 298,587,072 | 12,496 | 23,894 |
| nn | 43,538,648 | 1,482 | 29,378 |
| en | 39,155,566 | 1,249 | 31,349 |
| de | 15,073,597 | 258 | 58,424 |
| sv | 5,158,922 | 110 | 46,899 |
| fr | 4,882,246 | 96 | 50,856 |
| fi | 1,496,060 | 30 | 49,868 |
| la | 1,217,184 | 36 | 33,810 |
| nl | 321,846 | 8 | 40,230 |
| it | 302,329 | 7 | 43,189 |
| es | 230,586 | 19 | 12,136 |
| lv | 96,409 | 1 | 96,409 |
| hu | 68,266 | 2 | 34,133 |
| eo | 57,660 | 1 | 57,660 |
| hr | 55,655 | 2 | 27,827 |
| pt | 54,009 | 1 | 54,009 |
| cs | 44,232 | 3 | 14,744 |
| pl | 14,783 | 1 | 14,783 |
| uk | 11,272 | 6 | 1,878 |
| et | 10,591 | 1 | 10,591 |
| als | 2 | 1 | 2 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 1,986,034 | 64 | 32,559 |
| 2010 | 17,750,006 | 972 | 142,591 |
| 2000 | 57,854,935 | 2,975 | 199,443 |
| 1990 | 116,865,204 | 6,029 | 196,895 |
| 1980 | 40,494,244 | 1,532 | 265,011 |
| 1970 | 22,111,500 | 861 | 252,218 |
| 1960 | 18,124,576 | 480 | 376,502 |
| 1950 | 17,603,940 | 350 | 480,234 |
| 1940 | 29,395,155 | 546 | 509,948 |
| 1930 | 36,535,860 | 716 | 507,173 |
| 1920 | 52,070,672 | 1,087 | 483,201 |
| 1910 | 63,920,279 | 1,255 | 501,227 |
| 1900 | 61,593,361 | 1,164 | 525,050 |
| 1890 | 88,616,464 | 1,814 | 485,506 |
| 1880 | 59,549,395 | 1,087 | 550,945 |
| 1870 | 26,541,488 | 634 | 406,854 |
| 1860 | 39,854,070 | 710 | 543,956 |
| 1850 | 55,078,195 | 864 | 635,165 |
| 1840 | 31,307,769 | 534 | 583,077 |
| 1830 | 18,377,415 | 374 | 479,400 |
| 1820 | 4,821,598 | 147 | 339,040 |
| 1810 | 1,013,747 | 58 | 130,214 |
A dump of Wikipedia downloaded 2021.06.20. The text is cleaned. The dump contains both Bokmål and Nynorsk, however this is marked in the corpus in the doc_type-tag. In addition there is automatic language detection by Fasttext that in most cases will overlap.
| License | Size | Words | Documents | Avg words per doc |
|---|
| CC BY-SA 3.0 | 1.0 GB | 140,992,663 | 681,973 | 206 |
View dataset statistics
| Source | Words | Documents | Words/Document |
|---|
| wikipedia_download_nbo | 113,329,119 | 535,461 | 211 |
| wikipedia_download_nno | 27,663,544 | 146,512 | 188 |
| Language | Words | Documents | Words/Document |
|---|
| no | 112,570,496 | 529,279 | 212 |
| nn | 25,606,369 | 136,572 | 187 |
| en | 1,138,003 | 3,550 | 320 |
| da | 565,459 | 6,928 | 81 |
| de | 340,884 | 1,555 | 219 |
| sv | 313,287 | 2,197 | 142 |
| fr | 118,657 | 360 | 329 |
| war | 52,692 | 139 | 379 |
| es | 47,415 | 274 | 173 |
| vi | 40,237 | 123 | 327 |
| ceb | 30,173 | 43 | 701 |
| it | 23,144 | 116 | 199 |
| ru | 20,264 | 162 | 125 |
| nl | 19,572 | 88 | 222 |
| pl | 13,917 | 75 | 185 |
| ko | 11,857 | 33 | 359 |
| uk | 11,347 | 94 | 120 |
| pt | 9,425 | 105 | 89 |
| hr | 7,126 | 19 | 375 |
| fi | 6,225 | 29 | 214 |
| hu | 5,236 | 28 | 187 |
| eo | 5,084 | 36 | 141 |
| ro | 3,977 | 13 | 305 |
| la | 2,908 | 8 | 363 |
| cs | 2,901 | 19 | 152 |
| pa | 2,870 | 1 | 2,870 |
| bo | 2,551 | 1 | 2,551 |
| ca | 2,489 | 12 | 207 |
| min | 2,095 | 1 | 2,095 |
| is | 1,977 | 5 | 395 |
| bn | 1,309 | 1 | 1,309 |
| be | 1,262 | 9 | 140 |
| az | 1,242 | 6 | 207 |
| sq | 1,058 | 6 | 176 |
| th | 878 | 4 | 219 |
| et | 844 | 7 | 120 |
| sh | 841 | 5 | 168 |
| fy | 730 | 4 | 182 |
| bg | 621 | 1 | 621 |
| as | 567 | 1 | 567 |
| lt | 525 | 7 | 75 |
| sr | 496 | 1 | 496 |
| sl | 398 | 7 | 56 |
| gl | 384 | 10 | 38 |
| zh | 358 | 4 | 89 |
| el | 349 | 3 | 116 |
| lv | 347 | 2 | 173 |
| ja | 236 | 5 | 47 |
| fa | 214 | 1 | 214 |
| hy | 196 | 3 | 65 |
| ur | 189 | 2 | 94 |
| ta | 187 | 1 | 187 |
| tr | 181 | 1 | 181 |
| kk | 133 | 1 | 133 |
| tl | 130 | 1 | 130 |
| nds | 112 | 1 | 112 |
| wa | 61 | 3 | 20 |
| km | 43 | 1 | 43 |
| ar | 35 | 1 | 35 |
| id | 23 | 2 | 11 |
| lrc | 20 | 1 | 20 |
| sk | 15 | 1 | 15 |
| eu | 15 | 1 | 15 |
| ga | 12 | 1 | 12 |
| pms | 8 | 1 | 8 |
| ms | 7 | 2 | 3 |
| Decade | Words | Documents | Words/Document |
|---|
| 2020 | 140,992,663 | 681,973 | 206 |
Please also note that there are Norwegian Web Corpora available, most notably the OSCAR and the MC4 dataset. Both these sets have a Norwegian Bokmål and a Norwegian Nynorsk subset. MC4 is the largest set (70GB) but is fairly uncleaned with a significant part of machine generated text. The OSCAR set is smaller (?) but is better clened. Almost the entire OSCAR corpus is contained in MC4, so it is not recommented adding both. It might also be an alternative to add just a portion of MC4 because of its size and relatively low quality.
We are unable to redistribute these sets but the datasets are in the same HuggingFace Dataset format and can easily be combined with these sets.