Patarakin: 1 версия импортирована

2022-07-21T08:33:17Z

1 версия импортирована

← Предыдущая версия	Версия от 11:33, 21 июля 2022
(нет различий)

scratch>NFlex23: /* Tokenizer */ Added a comment for clarification

2022-07-08T19:30:20Z

Tokenizer: Added a comment for clarification

Новая страница

This tutorial teaches how to create a simple [[Programming Language|programming language]] that uses [[wikipedia:S-Expression|s-expressions]] for its instructions. Here is an example snippet of the language:
<syntaxhighlight>
(say (+ (* 4 7) 2))
</syntaxhighlight>
[[projects:664143277|This project]] contains the completed code if you don't want to go through the whole tutorial.

==Prerequisites==
You will need to create these [[Variables]].
* <scratchblocks>(i)</scratchblocks>
* <scratchblocks>(accumulator)</scratchblocks>
* <scratchblocks>(token)</scratchblocks>

Also create these [[Lists]].
* <scratchblocks>(tokens :: list)</scratchblocks>
* <scratchblocks>(instructions :: list)</scratchblocks>
* <scratchblocks>(operator stack :: list)</scratchblocks>
* <scratchblocks>(stack :: list)</scratchblocks>

==Tokenizer==
The [[Programming Language (project type)#Lexer|tokenizer]] (also known as a lexer) converts a program into a list of [[wikipedia:Lexical Analysis#Token|tokens]], or bits of the original program like words, numbers, and parentheses.
{{note|All of the following custom blocks should have "Run without screen refresh" checked.}}
<scratchblocks>
define tokenize (program)
set [i v] to [1]
set [accumulator v] to []
delete all of [tokens v]
repeat until <(i) > (length of (program))>
set [token v] to (letter (i) of (program))
if <[() ] contains (token)> then // Make sure to include a space after "()" or it won't work correctly
if <not <(accumulator) = []>> then
add (accumulator) to [tokens v]
set [accumulator v] to []
end
if <[()] contains (token)> then
add (token) to [tokens v]
end
else
set [accumulator v] to (join (accumulator) (token))
end
change [i v] by [1]
end
if <not <(accumulator) = []>> then
add (accumulator) to [tokens v]
end
</scratchblocks>

==Parser==
The [[Programming Language (project type)#Parser|parser]] takes in the tokens and builds a [[wikipedia:Parse tree|parse tree]], or in the case of this language, a simple form of [[wikipedia:Assembly language|assembly]].
<scratchblocks>
define parse
delete all of [instructions v]
delete all of [operator stack v]
repeat until <(length of [tokens v]) = [0]>
set [token v] to (item [1] of [tokens v])
delete [1] of [tokens v]
if <(token) = [(]> then
add (item [1] of [tokens v]) to [operator stack v]
delete [1] of [tokens v]
else
if <(token) = [)]> then
add (item (length of [operator stack v]) of [operator stack v]) to [instructions v]
delete (length of [operator stack v]) of [operator stack v]
else
add [push] to [instructions v]
add (token) to [instructions v]
end
end
end
if <(length of [operator stack v]) > [0]> then
say [Unclosed parenthesis] for [2] seconds
stop [all v]
end
</scratchblocks>

==Evaluator==
The last step is evaluating (also known as [[Programming Language (project type)#Interpretation|interpreting]]) the parsed instructions. Without this step, all we would have is a list of instructions which is pretty useless on its own.
<scratchblocks>
define evaluate
set [i v] to [1]
delete all of [stack v]
repeat until <(i) > (length of [instructions v])>
run instruction (item (i) of [instructions v])
end

define run instruction (operator)
if <(operator) = [push]> then
add (item ((i) + [1]) of [instructions v]) to [stack v]
change [i v] by [2]
stop [this script v]
end
if <(operator) = [say]> then
say (item (length of [stack v]) of [stack v]) for [2] seconds
change [i v] by [1]
stop [this script v]
end
if <(operator) = [+]> then
replace item ((length of [stack v]) - [1]) of [stack v] with ((item ((length of [stack v]) - [1]) of [stack v]) + (item (length of [stack v]) of [stack v]))
delete (length of [stack v]) of [stack v]
change [i v] by [1]
stop [this script v]
end
if <(operator) = [-]> then
replace item ((length of [stack v]) - [1]) of [stack v] with ((item ((length of [stack v]) - [1]) of [stack v]) - (item (length of [stack v]) of [stack v]))
delete (length of [stack v]) of [stack v]
change [i v] by [1]
stop [this script v]
end
if <(operator) = [*]> then
replace item ((length of [stack v]) - [1]) of [stack v] with ((item ((length of [stack v]) - [1]) of [stack v]) * (item (length of [stack v]) of [stack v]))
delete (length of [stack v]) of [stack v]
change [i v] by [1]
stop [this script v]
end
if <(operator) = [/]> then
replace item ((length of [stack v]) - [1]) of [stack v] with ((item ((length of [stack v]) - [1]) of [stack v]) / (item (length of [stack v]) of [stack v]))
delete (length of [stack v]) of [stack v]
change [i v] by [1]
stop [this script v]
end
... // You can add your own instructions to make the language even better!
</scratchblocks>

==Putting It All Together==
<scratchblocks>
when gf clicked
tokenize [(say (+ (* 4 7) 2))] :: custom
parse :: custom
evaluate :: custom
</scratchblocks>
When you run the program, the sprite should say "30".

==Final Thoughts==
While the language works, it could be greatly improved. Here are some things you could do to improve it:
* More functions, like <code>ask</code> and <code>say-for-seconds</code>.
* Better [[wikipedia:Exception handling|error handling]]. Currently, the language doesn't care if you do something wrong, like <code>(say 1 2 3)</code>.
* [[wikipedia:Control flow|Control flow]] functions, like <code>if</code>, <code>while</code>, and <code>for</code>. To implement these you would need to create some sort of [[wikipedia:Control flow#Labels|jumping system]] to move around the program.
A project with all of the features above can be viewed [[projects:656513485|here]].

[[Category:Math Tutorials]] [[Category:Scripting Tutorials]]

Creating a Programming Language - История изменений

Patarakin: 1 версия импортирована

scratch>NFlex23: /* Tokenizer */ Added a comment for clarification