Is tokenizers safe to use?

tokenizers has no known vulnerabilities on the latest version (0.3.0). DepScope rates its health at 46/100 (high risk).

What is the latest version of tokenizers?

The latest version of tokenizers is 0.3.0, released 2022-12-22.

Does tokenizers have known vulnerabilities?

No known vulnerabilities on the latest version of tokenizers.

Is tokenizers deprecated?

No, tokenizers is actively maintained. Latest release: 0.3.0 (2022-12-22).

What are alternatives to tokenizers?

DepScope has no curated alternatives for tokenizers at this time.

What license does tokenizers use?

tokenizers is licensed under MIT + file LICENSE.

tokenizers

cranv0.3.0

Fast, Consistent Tokenization of Natural Language Text. Convert natural language text into tokens. Includes tokenizers for shingled n-grams, skip n-grams, words, word stems, sentences, paragraphs, characters, shingled characters, lines, Penn Treebank, regular expressions, as well as functions for counting characters, words, and sentences, and a function

License MIT + file LICENSE0 versions1 maintainers3 deps11,480 weekly dl

ropensci/tokenizers

/ 100

Health

safe to use

[email protected] is safe to use (health: 46/100)

Health breakdown0 – 100

0/25

maintenance

10/20

popularity

25/25

security

9/15

maturity

2/15

community

Vulnerabilities

none known

Health History

Dependency Tree

License Audit

Dependencies (3)

stringi Rcpp SnowballC

API access

Get this data programmatically — free, no authentication.

curl https://depscope.dev/api/check/cran/tokenizers

First published · 2022-12-22 10:32:39

Last updated · 2022-12-22T07:50:02+00:00