How to copy all the words in a paragraph and add each word # on delphi

Question

Using Delphi 7, if I have some text like that shown in the lefthand window below, how could I extract all the words and punctuation in a paragraph and copy them to another window like that on the right, followed by a #?

Answer 1

If I'm understanding you correctly, you need what's called a "tokeniser" or "lexer".

D7 comes with one built-in, in the Classes unit, misleadingly called TParser (misleadingly because parsing normally means the "grammatical analysis" step which may follow the tokenisation of the text, as happens f.i. in the operation of a compiler).

Anyway, iirc, Delphi's TParser was intended to do things like process the text of DFM files, so will not necessarily split the text up exactly as you want, but it's a start. For example, when it tokenises ":=", it will return the ":" and "=" as two separate tokens, but, of course you are free to join them up again when NextToken/TokenString return these in succession. Btw, there are several alternative ways of implementing a tokeniser using classes in the freeware Jedi JCL and JVCL libraries, f.i.

If the text window on the left of your q is in your own app, code like the following may do what you want:

procedure TForm1.Tokenise;
var
  SS : TStringStream;
  TokenText : String;
  Parser : TParser;
begin
  SS := TStringStream.Create(Memo1.Lines.Text);
  Parser := TParser.Create(SS);

  try
    while Parser.Token <> #0 do begin
      TokenText :=  Parser.TokenString;
      Memo2.Lines.Add(TokenText + '#');
      Parser.NextToken;
    end;
  finally
    Parser.Free;
    SS.Free;
  end;

end;

If the text window is in another app, you would need a method of retrieving the text from it, too, of course.