TokenCounter of a string displaying not desirable results

  • Thread starter TheMathNoob
  • Start date
  • Tags
    String
In summary: Token function in the given code that causes it to count two tokens instead of one when there are trailing spaces in the first line of the file. This issue affects the accuracy of the program's error checking process.
  • #1
TheMathNoob
189
4

Homework Statement


I am currently coding a program which reads a file and makes a graph based on the arrangement of the data in the file. The first line of the file stands for the number of vertices and the rest of the lines stand for the edges
something like this:
3
1 2
2 3
3 4
In the first line when I add a second number such as 4, so the first line looks like 3 4, my parseN function is supposed to display an error saying the first line is wrong hence there should be just one number. To achieve this, I use a counterToken function which counts the number of tokens in the first line. When I add this second number, the program works fine displaying the error, but then when I delete this second number and leave the text cursor 2 spaces away from the string in this case "3", my counterToken function begins to count 2 tokens all the time in the first line.

Homework Equations


Code:
int ParseN(char line[])
{
    int verticesN;
    int NumberOfTokens=CountTokens(line);
    if(NumberOfTokens!=1)
        {
            printf("bad firstline\n");
            fflush(stdout);
            exit(0);
        }

    char* pt=strtok(line," -");
     verticesN=atoi(pt);
    return verticesN;
}

int CountTokens(char line[])
{
    char clone[sizeof(line)];
    strcpy(clone,line);

          const char s[2] = " -";
          char *token;
          int number=0;
          token = strtok(clone,s);
          while( token != NULL )
          {
              number++;
             token = strtok(NULL,s);

          }
          free(token);

          return number;
}

The Attempt at a Solution

 
Last edited by a moderator:
Physics news on Phys.org
  • #2
I think you're saying that if the first line contains a number with trailing spaces, your routine returns a count of two or more numbers?
 
  • #3
NascentOxygen said:
I think you're saying that if the first line contains a number with trailing spaces, your routine returns a count of two or more numbers?
Yes
 

FAQ: TokenCounter of a string displaying not desirable results

1. How can I fix the TokenCounter of a string displaying not desirable results?

There are a few steps you can take to fix this issue. First, check to make sure that your code is correctly counting the tokens in the string. You may need to adjust your code to account for special characters or punctuation. Secondly, check the source of the string and make sure it is in the correct format. If the string is not formatted correctly, the TokenCounter may not work properly. Finally, make sure that your TokenCounter is using the correct algorithm and that it is efficient enough to handle large strings.

2. Why is my TokenCounter giving me incorrect results?

There could be a few reasons for this. First, double check your code to make sure there are no logical errors. You may also want to check the source of the string to make sure it is in the correct format. Additionally, if you are using a pre-made TokenCounter algorithm, make sure it is compatible with your programming language and that you are using it correctly.

3. Can I customize the TokenCounter to ignore certain words or characters?

Yes, you can customize the TokenCounter to ignore specific words or characters. This can be useful if you want to exclude common words like "the" or punctuation marks from your token count. You will need to adjust your code to include these customizations.

4. Is there a limit to the length of the string that the TokenCounter can handle?

The length of the string that the TokenCounter can handle will depend on the efficiency of your code and the resources available on your computer. Generally, there is no set limit, but extremely large strings may cause performance issues.

5. Can I use the TokenCounter for languages other than English?

Yes, you can use the TokenCounter for languages other than English. However, you may need to adjust the algorithm or add additional code to handle special characters or punctuation in different languages. It is important to consider the specific rules and structure of the language you are working with when using a TokenCounter.

Similar threads

Back
Top