Part #2 of our series on building web console for Elixir & Phoenix applications with Phoenix LiveView.
In this series:
- Part 1: Project set up and simple code executor
- Part 2: Making imports and macros work
- Part 3: Managing multiple supervised sessions
- Part 4: Making it real-time
Video version
Watch the video version below or or directly on YouTube for a complete step-by-step walkthrough.
The road so far
The starting point of today’s episode is the result of our “part 1 blog post / video”. We have working environment to execute basic Elixir code, but it’s not yet close to being usable in real life.
Among other things, the version we built so far lacks the ability to use import or require statements, and it crashes whenever an exception in source code is encountered, re-setting the whole user interface, including output and commands history.
Separation of concerns
We start off, however by separating the code executing back-end code from LiveView front-end.
Each Elixir or Erlang application can start it’s own supervision tree. Our backdoor library is no exception here, and, in fact, it already declares and starts one. If you look into it’s application callback module, you will see that it starts a DynamicSupervisor already. We don’t use it just yet, but we will, so I’ll leave that in, but will add a line that also starts single process of Backdoor.CodeExecutor, and registers it’s name so other processes can find it’s PID easily:
# lib/backdoor/application.ex
defmodule Backdoor.Application do
@moduledoc false
use Application
def start(_, _) do
children = [
{Backdoor.CodeRunner, name: Backdoor.CodeRunner}, # <-- Add this line!
{DynamicSupervisor, name: Backdoor.DynamicSupervisor, strategy: :one_for_one}
]
Supervisor.start_link(children, strategy: :one_for_one)
end
endBackdoor.CodeRunner will be a simple GenServer-based code executor, and it’s responsibilities will be limited to taking input from the user, parsing it, executing the code and reporting back to the caller process using a defined interface. It will also keep track of bindings that we learned about in part 1, but also environment that we will learn about today.
In the future, we will be starting multiple CodeRunner processes, one per each session that user starts. For now, however, our library will only start single CodeRunner, which will be supervised and started whenever the application starts.
Our backdoor library is fairly similar to IEx itself, and it makes sense that IEx provides a module very similar to our CodeRunner - it’s IEx.Evaluator gives us some hints how to solve problems we are facing.
In particular, the problem of being unable to use import or require in a meaningful way, is being addressed by initializing and keeping track of env variable. IEx initializes it with function :elixir.env_for_eval(file: "iex"), and we can have a peek what’s inside:
iex(1)> IO.inspect(env, structs: false, limit: :infinity)
%{
__struct__: Macro.Env,
aliases: [],
context: nil,
context_modules: [],
contextual_vars: [],
current_vars: {%{}, false},
file: "iex",
function: nil,
functions: [
{Kernel,
[
!=: 2,
!==: 2,
*: 2,
+: 1,
+: 2,
++: 2,
-: 1,
-: 2,
--: 2,
/: 2,
<: 2,
<=: 2,
==: 2,
===: 2,
=~: 2,
>: 2,
>=: 2,
abs: 1,
apply: 2,
apply: 3,
binary_part: 3,
bit_size: 1,
byte_size: 1,
ceil: 1,
div: 2,
elem: 2,
exit: 1,
floor: 1,
function_exported?: 3,
get_and_update_in: 3,
get_in: 2,
hd: 1,
inspect: 1,
inspect: 2,
is_atom: 1,
is_binary: 1,
is_bitstring: 1,
is_boolean: 1,
is_float: 1,
is_function: 1,
is_function: 2,
is_integer: 1,
is_list: 1,
is_map: 1,
is_map_key: 2,
is_number: 1,
is_pid: 1,
is_port: 1,
is_reference: 1,
is_tuple: 1,
length: 1,
macro_exported?: 3,
make_ref: 0,
map_size: 1,
max: 2,
min: 2,
node: 0,
node: 1,
not: 1,
pop_in: 2,
put_elem: 3,
put_in: 3,
rem: 2,
round: 1,
self: 0,
send: 2,
spawn: 1,
spawn: 3,
spawn_link: 1,
spawn_link: 3,
spawn_monitor: 1,
spawn_monitor: 3,
struct: 1,
struct: 2,
struct!: 1,
struct!: 2,
throw: 1,
tl: 1,
trunc: 1,
tuple_size: 1,
update_in: 3
]}
],
lexical_tracker: nil,
line: 1,
macro_aliases: [],
macros: [
{Kernel,
[
!: 1,
&&: 2,
..: 2,
<>: 2,
@: 1,
alias!: 1,
and: 2,
binding: 0,
binding: 1,
def: 1,
def: 2,
defdelegate: 2,
defexception: 1,
defguard: 1,
defguardp: 1,
defimpl: 2,
defimpl: 3,
defmacro: 1,
defmacro: 2,
defmacrop: 1,
defmacrop: 2,
defmodule: 2,
defoverridable: 1,
defp: 1,
defp: 2,
defprotocol: 2,
defstruct: 1,
destructure: 2,
get_and_update_in: 2,
if: 2,
in: 2,
is_nil: 1,
is_struct: 1,
match?: 2,
or: 2,
pop_in: 1,
put_in: 2,
raise: 1,
raise: 2,
reraise: 2,
reraise: 3,
sigil_C: 2,
sigil_D: 2,
sigil_N: 2,
sigil_R: 2,
sigil_S: 2,
sigil_T: 2,
sigil_U: 2,
sigil_W: 2,
sigil_c: 2,
sigil_r: 2,
sigil_s: 2,
sigil_w: 2,
to_char_list: 1,
to_charlist: 1,
to_string: 1,
unless: 2,
update_in: 2,
use: 1,
use: 2,
var!: 1,
var!: 2,
|>: 2,
||: 2
]}
],
module: nil,
prematch_vars: :warn,
requires: [Application, Kernel, Kernel.Typespec],
tracers: [],
unused_vars: {%{}, 0},
vars: []
}So this thing is a Macro.Env struct, which keeps track of imported functions, requires, imports and a bunch of other things we probably need to have. Sweet.
The problem in our code is not only that we don’t declare nor use this struct, it’s also that we can’t pass it to Elixir’s own Code.eval_quoted/3 that we’ve been using, as it doesn’t seem to take it as an argument.
Having another look at IEx’s Evaluator module reveals that it’s using :elixir.eval_forms/3 instead, which takes abstract syntax tree, bindings and environment as it’s arguments. Let’s switch our code to use the same, but first let’s build it up as a GenServer:
# lib/backdoor/code_runner.ex
defmodule Backdoor.CodeRunner do
use GenServer
# Public API
def start_link(opts) do
GenServer.start_link(__MODULE__, [], opts)
end
# Callbacks
@impl true
def init(_) do
{:ok, %{bindings: [], env: init_env()}}
end
# private
defp init_env do
:elixir.env_for_eval(file: "backdoor")
end
endAfter adding the module above we can start or dev environment with mix dev, and the extra process should start correctly. It doesn’t do anything just yet, nor it has any public API that allows sending code to execute in it. Let’s fix that by adding execute/2 function:
# Public API
...
def execute(runner, code) do
GenServer.call(runner, {:execute, code})
end
...
# private
...
@impl true
def handle_call({:execute, code}, _from, state) do
try do
{result, bindings, env} = do_execute(code, state.bindings, state.env)
{:reply, {:ok, result}, %{state | bindings: bindings, env: env}}
catch
kind, error ->
{:reply, {:error, kind, error, __STACKTRACE__}, state}
end
end
...
# private
...
defp do_execute(code, bindings, env) do
{:ok, ast} = Code.string_to_quoted(code)
:elixir.eval_forms(ast, bindings, env)
endWe also wrapped the code parsing and exeution functions into try/catch clause, and we returin {:error, kind, error, stacktrace} tuple in case exception has been raised. The resulting module source code is fairly short and sweet, and does what we need it to do - but it doesn’t get used just yet.
We need to clean up our user interface LiveView of any code execution and bindings tracking logic, and make it call our Backdoor.CodeRunner process instead. The resulting LiveView module has the handle_event/3 calling the CodeRunner:
@impl true
def handle_event("execute", %{"command" => %{"text" => command}}, socket) do
formatted_result_or_error =
with pid <- GenServer.whereis(Backdoor.CodeRunner),
{:ok, result} <- Backdoor.CodeRunner.execute(pid, command) do
inspect(result)
else
{:error, kind, error, stack} ->
format_error(kind, error, stack)
end
output = socket.assigns.output ++ ["backdoor> " <> command] ++ [formatted_result_or_error]
{:noreply,
socket
|> push_event("command", %{text: ""})
|> assign(output: output)}
end
defp format_error(kind, error, stack) do
Exception.format(kind, error, stack)
endIt also formats errors if they are encountered and outputs results, so the user can read exceptions with a stacktraces! Sweet!
Good but not yet perfect
I feel like at this moment we have the basics of code execution covered. There are, however, two more things we desperately need to fix in order to make this thing usable:
- Redirect standard output / error output to our web interface. Currently
Logger.*orputsstatements do not crash, but the output is seen in the terminal where you started the app and not in the web UI. - Allow multi-line Elixir statements to be written by the user and executed. Currently the console is only able to execute one-liners. This is not very useful if you need to write a module or function.
We will fix both of these issues in the next episode, and then start on either adding some tests (we haven’t written any!) or start implementing multiple sessions support.
Notes
Watch the screencast directly on YouTube
You can find this and future videos on our YouTube channel.
The full code from this episode can be found on GitHub
Post by Hubert Łępicki
Hubert is partner at AmberBit. Rails, Elixir and functional programming are his areas of expertise.