I went through the code for the Dashboard artifact (crud.py and Dashboard.ipynb) and updated the names of variables and functions to reflect the context of the new use case. I modified the MongoDB connection string to use a new database and require a user name and password, but rolled this change back (temporarily) to reduce the complexity of the rest of the development process; I have documented the secure connection string for later use. I modified the code in Dashboard.ipynb to display an appropriate logo, and I added labeled text boxes that accept user input, passed to the backend code when the user clicks on a submit button. A callback function then stores the user input in a collection of variables. Having integrated the Cartpole artifact code into the Dashboard artifact (broken into an imported class file and a modified cartpole() function), the collection of user input variables are passed as arguments to the cartpole() function. These arguments represent hyperparameters of the underlying deep-Q reinforcement learning algorithm, which can be tuned to improve model performance. The code in score_logger.py, was modified to write performance relevant data into a file named metrics.csv, along with all of the hyperparameters submitted by the user. After each training session this data can be written to the MongoDB database using the write() or writes() functions, which I added to the crud.py module. Currently, from Jupyter Notebook, running the Dashboard.ipynb file displays an unpopulated data table, and the relevant chart functionality needs to be added. When a user enters hyperparameter values and clicks on the “Submit” button, it initiates the training of the cartpole learning algorithm.
I encountered an issue that has been very difficult to resolve, involving memory consumption. The learning algorithm holds a huge amount of data in RAM and I repeatedly encountered Jupyter Notebook kernel failures due to what seems like memory buffer limits. I began to troubleshoot by running the Cartpole program in isolation, and encountered the same issues. I spent a lot of time surveying dependency version compatibility and testing different Anaconda environment configurations, adjusted system settings to see if I could make the code run better, tested different combinations of hyperparameters, and converted the cartpole.ipynb to a .py file to run in the terminal, all to eliminate any ambiguities going forward. A major issue is that memory shortages do not occur until the learning algorithm has been training for hours, so iterative performative feedback is slow-going. Ultimately, I need the program to finish running in order to create complete data to store in the database, which can then be loaded into a Pandas data frame (or a memory safe alternative) and made available for interface driven query, filtering, and aggregation as well as graphical data visualizations. My current system has 16 GB of RAM, and I am also using 16 GB of Paging Memory (Swap) at the time of this writing. Using more paging memory can allow the memory requirements of the program but leads to very slow execution as run-time progresses. I have had the greatest success when greatly increasing my virtual memory on the order of 80 to 100 GB. I will need to examine whether there is any way to decrease RAM requirements but I think that it may be a memory intensive algorithm no matter what I do about it.
I met a large number of my enhancement goals, and addressed the feedback provided by my professor, but the sheer amount of time it takes to train and troubleshoot machine learning algorithms resulted in some of the final goals being temporarily tabled. I still plan on keeping my original plans for the program, but have a few questions and concerns. Having integrated the two artifacts, increased modularity, updated naming conventions, updating deprecated dependencies, and restructuring the code to reflect its new purpose, I have been reminded of the incredibly large amount of RAM required to run the machine learning model, and therefore the final artifact itself. To give some idea of this hardware requirement, the model only completed training successfully when I expanded my computer’s virtual memory to about 80 GB above my 16 GB of physical memory. Consequently, I will be researching about ways to reduce the amount of memory needed so that the final artifact can be useful to novices, as was originally intended. Also, while the mechanism for storing summary data about the performance resulting from a specific set of hyperparameters is currently in place, the functionality related to transferring this data from csv to the MongoDB database, and the data visualization methods still need to be completed.
I began by going through the code for the Dashboard artifact (crud.py and Dashboard.ipynb) and updating the names of variables and functions to reflect the context of the new use case. I modified the MongoDB connection string to use a new database and require a user name and password, but rolled this change back (temporarily) to reduce the complexity of the rest of the development process; I have documented the secure connection string for later use. I modified the code in Dashboard.ipynb to display an appropriate logo, and I added labeled text boxed that accept user input, passed to the backend code when the user clicks on a submit button. A callback function then stores the user input in a collection of variables. Having integrated the Cartpole artifact code into the Dashboard artifact (broken into an imported class file and a modified cartpole() function), the collection of user input variables are passed as arguments to the cartpole() function. These arguments represent hyperparameters of the underlying deep-Q reinforcement learning algorithm, which can be tuned to improve model performance. The code in score_logger.py, was modified to write performance relevant data into a file named metrics.csv, along with all of the hyperparameters submitted by the user. After each training session this data can be written to the MongoDB database using the write() or writes() functions, which I added to the crud.py module. Currently, from Jupyter Notebook, running the Dashboard.ipynb file displays an unpopulated data table, and the relevant chart functionality needs to be added. When a user enters hyperparameter values and clicks on the “Submit” button, it initiates the training of the cartpole learning algorithm.
This site was built using GitHub Pages You can find the source code for Jekyll at GitHub