Lua as a general-purpose scripting language?

Lua is a cool language, light-weight and extremely fast!

But the point is: Is performance so important for those
tasks you mentioned?

  • Renaming a bunch of files
  • Download some files from the web
  • Webscraping

You write those programs once, and run them once, too maybe.
Why do you care about performance so much for a run-once program?

For example:

  1. Cost 3 hours to write a C/C++ program, to handle data once, the program will take 1 hour to run.
  2. Cost 30 Minute to write a Python program to handle data once, the program will take 10 hours to run.

If you choose the first, you save the time to run the program,
but you cost your time to develop the program.

On the other hand, if you choose the second, you waste time to run
the program, but you can do other things when the program is
running. How about play World of Warcraft, kill monsters
with your warlock? Eat my D.O.T
! 😛

That’s it! Although Lua is not so difficult to write, everything about Lua is designed to be efficient.And what’s more, there are little modules for Lua, but there are so many modules for Python. You don’t want to port a C library for Lua just for a run-once program, do you? Instead, choose Python and use those module to achieve your task easily might be a better idea.

FYI: Actually, I have tried to use Lua to do webscraping,
but finally, I realized I do not have to care so much about language performance. The bottleneck of webscraping is
not on the performance of the language
. The bottleneck is on
network I/O, HTML parsing and multitasking. All I have to do
is make sure the program works and find the bottleneck.
Finally, I chose Python rather than Lua. There is so
many excellent Python modules; I have no reason to build my
own.

According to my experience about webscraping, I chose
Twisted for network I/O and lxml for html parsing as the backend
of my webscraping program. I have wrote an article for an introduction to this technology.

The best choice to grab data from websites: Python + Twisted + lxml

Hope this is helpful.

Leave a Comment