Question In C, how does strtok() break a string into tokens?

Please provide comments and solutions that are helpful to the author of this topic.

quentisa

Level 1
Thread author
Jan 6, 2023
11
Please explain how the strtok() method works. According to the documentation, it splits the string into tokens. I can't figure out what it does based on the handbook.
Code:
/* strtok example */
#include <stdio.h>
#include <string.h>

int main ()
{
  char str[] ="- This, a sample string.";
  char * pch;
  printf ("Splitting string \"%s\" into tokens:\n",str);
  pch = strtok (str," ,.-");
  while (pch != NULL)
  {
    printf ("%s\n",pch);
    pch = strtok (NULL, " ,.-");
  }
  return 0;
}
When the first while loop began, the contents of str were merely "this." I added watches on str and *pch to test its functionality. How did the following output appear on the screen?
 
  • Like
Reactions: [correlate]

harlan4096

Moderator
Verified
Staff Member
Malware Hunter
Well-known
Apr 28, 2015
8,729
I think more or less it's clear, checking the result of the compilation/execution:

1673282601357.png


 

Bot

AI-powered Bot
Apr 21, 2016
3,652
`strtok()` is a function from the C standard library that is used to break a string into a series of substring or tokens on the basis of a specified delimiter. Here is how it works:

In the example code you provided, the string `" - This, a sample string."` is passed to strtok() along with the specified delimiter `" ,.-"`. The delimiter is a string that contains all characters that can be used as delimiters. These characters will be used by strtok() to split the input string into a series of tokens.

Strtok() returns the first token in the input string, which in this case is "-". The first parameter is the string to be tokenized, and the second parameter is the set of delimiters that defines where tokens will be separated.

On a subsequent call to strtok() with a NULL string pointer as its first argument, the function resumes scanning the same string, and starts searching for the next token following the termination point of the previous one.

In the while loop, the function continues to call strtok() using a NULL pointer as the first argument until there are no more tokens left in the string which is indicated by a NULL return value.

So, the output you see on the screen is the individual tokens or substrings separated by the specified delimiter.

In the example code you provided, "Splitting string "- This, a sample string." into tokens:" is displayed as the first line of output. Then on the next line "- This" is printed, followed by "a", "sample" and "string". These are individual substrings or tokens defined by the delimiters specified.
 
  • Like
Reactions: Trident

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top