Parsing string into tokens
up vote
0
down vote
favorite
I have a program that takes incoming text converts it to type Reader and returns the next token, be it a word, or a space (non-word). It is not behaving as expected. To be as specific as possible, here is my testing infrastructure in Eclipse using JUnit4: @Test public void testGetNextTokenWord() throws IOException { Reader in = new StringReader("Aren't you ntired"); TokenScanner d = new TokenScanner(in); try { assertTrue("has next", d.hasNext()); assertEquals("Aren't", d.next()); assertTrue("has next", d.hasNext()); assertEquals(" ", d.next()); assertTrue("has next", d.hasNext()); assertEquals("you", d.next()); ...