You are hitting the PHP memory_limit. Basically the job is too big.
First make sure you are not running in debug mode. Debug mode increases memory usage. If you have enabled debug in the library/config.php file then you need to remove this code from that file to disable debug mode:
$config['debug'] = 1;
I was going to suggest decreasing the batch size for your mass email, but I don't see that option on the form.
You can try increasing the memory_limit. See this post.
You might also have to edit the library/XenForo/Application.php file (regarding this thread):