Question In C, how does strtok() break a string into tokens?

[quentisa]

Please explain how the strtok() method works. According to the documentation, it splits the string into tokens. I can't figure out what it does based on the handbook.

/* strtok example */
#include <stdio.h>
#include <string.h>

int main ()
{
  char str[] ="- This, a sample string.";
  char * pch;
  printf ("Splitting string \"%s\" into tokens:\n",str);
  pch = strtok (str," ,.-");
  while (pch != NULL)
  {
    printf ("%s\n",pch);
    pch = strtok (NULL, " ,.-");
  }
  return 0;
}

When the first while loop began, the contents of str were merely "this." I added watches on str and *pch to test its functionality. How did the following output appear on the screen?

 

[harlan4096]

I think more or less it's clear, checking the result of the compilation/execution:

C library - strtok() function

The C Library strtok() function is used for tokenizing strings. These strings are a set of tokens using delimiters/separators characters. In general, tokens are usually words, phrases, or individual characters within a string.

www.tutorialspoint.com

 

[Bot]

`strtok()` is a function from the C standard library that is used to break a string into a series of substring or tokens on the basis of a specified delimiter. Here is how it works:

In the example code you provided, the string `" - This, a sample string."` is passed to strtok() along with the specified delimiter `" ,.-"`. The delimiter is a string that contains all characters that can be used as delimiters. These characters will be used by strtok() to split the input string into a series of tokens.

Strtok() returns the first token in the input string, which in this case is "-". The first parameter is the string to be tokenized, and the second parameter is the set of delimiters that defines where tokens will be separated.

On a subsequent call to strtok() with a NULL string pointer as its first argument, the function resumes scanning the same string, and starts searching for the next token following the termination point of the previous one.

In the while loop, the function continues to call strtok() using a NULL pointer as the first argument until there are no more tokens left in the string which is indicated by a NULL return value.

So, the output you see on the screen is the individual tokens or substrings separated by the specified delimiter.

In the example code you provided, "Splitting string "- This, a sample string." into tokens:" is displayed as the first line of output. Then on the next line "- This" is printed, followed by "a", "sample" and "string". These are individual substrings or tokens defined by the delimiters specified.