Debugging, Logging and Software Development
“ If debugging is the process of removing software bugs, then programming must be the process of putting them in. ”
Edsger Dijkstra
During development of new features and integration there of a developer can be found staring at log data as it pours on his screen for errors, observing variable values and if they are in sync with expectation. Developers spends more time confronting, debugging and fixing software issues during development of software rather then after it has been deployed ( hopefully ).
Spectrum is to logging what sqlite3 is to databases. It’s a standalone logging server plus log viewer with filtering capabilities. It scales to multiple logging streams with endpoint being a file residing on filesystem, REST API endpoint, Syslog, UDPStream, WebSocketStream.
Let us give Spectrum a spin and see how it works.
Create Account
Visit https://www.devspectrum.com/register/ and register your account. This includes buying spectrum right away and sadly there is no trial period. Disclaimer we were provided access to the software by Spectrum for this product review.
Download Build and Registration Keys
Once your account is created the dashboard https://www.devspectrum.com/dashboard/ lists everything you need to get started
a) Copy Registration code somewhere secure,
b) Click “Download Latest Release” and select the operating system of your choice i.e. Linux, Windows, OSX.
c) Once the build has been downloaded extract the archive. (We are using the Linux build for this article). Execute Spectrum binary and you are greeted with a popup for registration code. Enter your Registration code from step 1 and we are in business.
Spectrum allows developer to organize logging into projects. Each project can have multiple streams. Each logging stream in turn can accommodates multiple levels of logging. Well that’s quite a mouthful but below examples will clear it.
File Based Stream
To fans of doing tail -f logfile.log on terminal this is for you. You wouldn’t have to worry about scrollback buffer. Simply create an endpoint with File as an option and select the log file.
Syslog and File based Stream
import logging
import logging.handlers
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
handler = logging.handlers.SysLogHandler(address = '/dev/log')
formatter = logging.Formatter('%(module)s.%(funcName)s: %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
def dummylogs():
logger.debug('this is debug')
logger.critical('this is critical')
if __name__ == '__main__':
dummylogs()
The entry level barrier to get spectrum up and running is very low, that’s why I called it sqlite3 of databases. Having the ability to search is very helpful too. e.g. “Traceback” shows the Python errors while the program executed.
REST based Stream
Spectrum can run as a standalone rest server binding to localhost and a port number of your choice. Python program can stream log messages to the server using spectrum-python library installable using pip.
pip install spectrum-python
Before we dive into code, it’s worth noting that a simple JSON HTTP POST to http://IP:PORT is all that’s needed to log messages to Spectrum in REST mode.
import requests
url = "http://localhost:9000"
data = {"id": "863ef2c3ea134841b87072", "level": "ERROR", "message": "The daemon process has generated an error code 555", "number": 253, "sublevel": "DAEMON", "timestamp": "2016-02-22 11:12:39.873108" }
headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
r = requests.post(url, data=json.dumps(data), headers=headers)
as long as your json has the keys id, level, sublevel, message, number, sublevel and timestamp you will be able to successfully log your message.
import logging
from spectrum.handlers import Spectrum as SpectrumAPI
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
spectrum = SpectrumAPI('Main-Process')
formatter = logging.Formatter('%(module)s.%(funcName)s: %(message)s')
spectrum.setFormatter(formatter)
logger.addHandler(spectrum)
def dummylogs():
logger.debug('this is debug')
logger.critical('this is critical')
if __name__ == '__main__':
dummylogs()
Conclusion
Pros
- Minimal Viable logger with a decent UI
- Easy to install, integrate and use
- Works out of the box for Python
- Ability to accommodate N no of projects with N no of streams and loggers.
Cons
- Search bar could have access to filters to toggle and not overlap the first line,
- File based logging method froze and crashed on us couple of times. Had to remove .spectrum directory in home folder and reinstall. This could be attributed to the fact that spectrum is a chrome app and not good at keeping up with filesystem IO. The software was writing an MB worth of data to the log file per second.