Wednesday, December 4, 2019

Release 0.4 - OpenClimateFix and the Github API

And so it ends...

As my course winds to a close and I finish up my assignments, I have reflected on everything I've accomplished in the last three months. I'm proud of what I've done. I've made 6 pull requests in public repos, 4 of which were merged, 1 of which is dead, and 1 is yet to be merged. I've looked at hundreds of issues in dozens of languages, learned so many things it's taken me 15 or 16 blog posts to contain them all. I am most proud of my work in this pull request, although on the surface it looks rather simple.

The Issue

The issue is built upon my previous pull request for Release 0.3, which is why I thought it would be perfect for 0.4. It is a new page on the website that includes the Contributor Showcase component I built for Release 0.3, as well as a paragraph on how to get involved, a couple of links, and a link to a github search for issues. It also includes a list of the repositories owned by OpenClimateFix. Another part of this issue is to attempt to add the Github API to the Get Involved page, to load the issues instead of clicking on the link. This is what I spent approximately 20-30 hours researching, learning, and testing before I found that I could not do it in this pull request. I figure what better way to use that knowledge than to impart it on the readers of this blog.

Why I couldn't use Github API

In order to use Github API, you can do 3 things:
  1. You can use a personal access token, which provides access to the Github API through your Github account. 
  2. You can set up a Github App, which means setting up a server to host a Github App to fetch data from the API.
  3. You can authenticate using a OAuth app in a similar manner.
What you can't do is use a personal access token on a public website. Everywhere I researched said explicitly that the point of the access token is for testing and maintaining your own repositories, and it is meant to be temporary as it only lasts for a year. It also can be used instead of a github password, which would give others access to my account and private repositories. The other 2 options would have to be done by OpenClimateFix, and would be a project on their own to get set up. The simplest solution is to just make a good Github search query link, and provide links to some of the repositories staticly.

Setting up the environment

Luckily, I have done a few PR's for OCF now, so I have the environment set up. I looked at many new packages and libraries to use the Github API, but in the end I had nothing to add. For your sake (the readers of this blog), I will include what I needed to get the Github API running. If I was making my own app, I would probably fetch the data differently, and not use Gatsby, but this is what I got working with the personal access token:
Then all I had to do was add the Github API query to my code, similarly to how I used a query in Release 0.3. The following code shows the query: 

query MyQuery {
  github {
    repository(name: "repository-name", owner: "owner") {
      id
      description
      issues(labels: "good first issue", states: OPEN, first: 10) {
        edges {
          node {
            id
            title
            url
          }
        }
      }
    }
  }
}

When I realized that personal access token was not going to be an option, I had to go with the backup plan.

The Solution (Pull Request)

The backup plan was to include a link using a Github search url to define open, good first issues available from all openclimatefix repositories. Here are the steps I went through to create the new page.
  1. Create a file named get-involved.jsx under the pages folder. This automatically creates a page in Gatsby, and routes it to the root uri followed by the filename. If the file was in a folder, it would default to `root/folder/jsx-file`. Classes are also added to the routes, so if I wanted to put all of my classes into one file I could, and have several routes. 
  2. Write the code to create and return the Gatsby html template.
  3. Add the existing code from the Get Involved section on the website. This includes the Contributor Showcase component I created for Release 0.3, as well as some links, and I added the new link for good first issues. I also left OCF some room to write their own message on the page.
  4. Format everything to fit prettier and eslint standards. The repository does not run eslint tests on compilation or on commit, but only in the pull request itself. Therefor, any errors are difficult to catch until you have made a pull request. It took several tries to fix the lint errors.
  5. Wait for a Review. I am currently in this stage, and will post an update if I hear anything back from OCF.
The Final Product

Overall it was a fairly simple solution that I spent so many hours researching for. I'm glad that through this blog post, my research and experience might help someone else get the Github API going in Gatsby. I found no guide suggesting where to go and what to do explicitly, so let this be a sort of guide. 

My Final Thoughts

As this course ends, I find myself disappointed and proud of myself. I am disappointed that I did not end up working on an issue for a larger company, like Microsoft, Google, or Amazon. I imagined my code up on some Google application, or as an addition to the newest stable build of Visual Studio Code. I would have liked to see a bug fix of mine in GIMP. But I ended up working on some amazing small projects, and that is what I am most proud of in this course. I found so many awesome projects, some of which I couldn't really help with and others I got heavily involved in. I am glad that I kept this blog (even though it is mandatory) because I know that I'll continue it, and maybe even take it with me to my next professional option course (on android app development). This will be my last blog post till after Christmas, so Happy Holidays to anyone reading!

Release 0.4 - Telescope's Documentation Problem

Why Documentation is Important

I have been working all weekend to digest the entirety of the Telescope repository for my course. I have been mentioning it for weeks now, but I'd like to go into detail now that I know more about it. Telescope essentially grabs blog posts from a feed centralized on a Seneca College website, and displays them on a front end app. This project has been moving very quickly, it only started a month ago and we are days away from a working prototype. But as quickly as it has been moving, the documentation has not been maintained at all, something I foresaw weeks ago when I made the CONTRIBUTING.md file for the project. I intended as developers added their code they would update the documentation, but they have not. 

When setting up the environment this weekend, I had great difficulty in understanding how and why things needed to be done. This is the point of documentation, to assist people in downloading and running the environment required to start development quickly. I am devoting my time on Telescope to solving these documentation issues, not just for the current developers but future developers as well. Now to the issue.

The Issue

Basically what needed to be done was to add documentation on docker, which has become a major component of the project, and how to set the .env variables. I have to talk to several people over the next few days in order to get this information, a process I have already started. Most of the information is readily available through the commit history of the env.example file. 

Background Information and Documentation

The following paragraphs detail what I learned in finding every avenue to install docker, and get the system running in different environments:

Virtualization

I had to learn how to properly set up a virtual machine. Mainly this was to get telescope running on a system that I could use. The main issue was that docker is not readily available on Windows Home Edition. At least so I thought at first. To even run docker, you need to have a virtualization capable system, and in some cases have to enable it in the BIOS. I decided to leave this out of my documentation as it is out of the scope of the project, and there are many guides online that can help you with this process. I even linked to a few guides and youtube videos in the documentation for assistance with certain steps.

Docker

I had to learn how Docker installation works, it's requirements, and what systems it can run on. This included finding alternatives for people who did not have access to the easier methods. For example, getting Docker for macOS is as simple as downloading Docker for Desktop. It will set up docker for you and comes with docker-compose pre-installed. On the other hand, Windows 10 Pro, Education, or Enterprise are required for Docker for Desktop to work. You need to either set up a virtual machine with Ubuntu Linux, get Windows Education Edition (an option freely available from Seneca), or get Docker Toolbox, which is an older version of Docker Desktop with less features.

On top of that, I had to learn how Docker works. It didn't occur to me that Docker essentially builds a miniature virtual machine to run build instructions in a consistent environment across devices, similar to my understanding of Java and it's environment. The difference is that Docker is fully customizable, and can use Docker images to use software that you would normally have to install natively, in our case this is Redis.

Docker-Compose

Docker-compose allows you to run docker applications by starting all services with one command. It is a separate install from docker on Linux distributions, and comes with Docker for Desktop on Windows and macOS. It is vital to the project as docker-compose is required to get the docker image of Redis running as a service on the right port. 

Chocolatey

Chocolatey is a package manager that was recommended to me by my professor as an easy way to download Redis natively. Part of my job was to investigate not using docker, and I found that although you can run the website locally without any major issues, it takes a lot of configuration and extra work rather than using docker. My recommendation was to entirely remove the native set up instructions and stick to docker.

The Review Process (Pull Request)

The review process took a lot longer than I initially anticipated. It took approximately 4 days to go from pull request to merge, and there were a lot of changes along the way. I came into the process with a limited understanding of exactly how everything fit together, and I have walked out knowing exactly how it all works. I was reviewed by Manekenpix and Silvyre, who are both extremely active in the project and have done many pull requests. Some of the concerns raise were as follows:
  • Code Syntax
  • Wording Problems
  • Grammar
  • Usage of sudo (when and why)
In total, between the three of us we made 35 comments, each of which added to the discussion of how much to include, and what should be excluded. This shows it's importance, as even extremely active community members take an interest in proper documentation.

I learned many things during the review process, including:
  • Docker Toolbox (Thanks to grommers00 on Github)
  • Docker for Desktop on Windows Education Edition (Thanks to MeisterRed on Github)
  • The proper usage of sudo
  • How to properly set up docker on Linux (with post install commands)

Overall Feelings on the Project

I feel like I made a good contribution to the project. Although I didn't contribute to the code itself, I can take solace knowing that future developers that work on the project in it's next iteration will be able to adapt and understand the code faster than I could. I have already had some people thank me for my contribution, saying it helped them get their environment set up better than they had it. I have enjoyed my time in this course, and this was a nice cherry on top. But there is still one more blog post to do after this one, and it contains my proudest moment in this course.

Thursday, November 28, 2019

Release 0.4 - Docker and The Search for An Issue

Progress So Far (Nov 28)

Telescope - I have 2 issues I am currently working on.
External PR - Still looking for an issue, but I have a potential one lined up.
Release 0.3 - Pull Request is still open, working on getting it closed.

Telescope

Getting Telescope Set Up

Until I discovered Windows Subsystem for Linux, I used a virtual machine for web development. I hated it. I hate virtual machines, at least my virtual machine. I have an excellent gaming PC at home that is one of the fastest I have ever used, for coding or otherwise. And instead of using it, I have to use a stupid slow virtual machine. WSL was amazing. I got the combined speed of my PC and the accessibility of linux commands and libraries. That is until telescope. 

Telescope requires docker, which cannot be run in WSL or on my Windows 10 Home Edition. It must be run in Ubuntu proper, MacOS, or Windows Professional. So I had to break out my stupid dumb annoying virtual machine and figure out installation. 

I was able to get the front end part running on my PC, as well as all the tests, so I am currently looking for more front-end related issues to work on. I am also working on a documentation fix and I have asked to look into a simple file name change as well. Of course, every reference to those files must also be changed, and tested.

External PR

Potential PR (if I get a response)

My potential external PR is to make the Get Involved page on the openclimatefix website. It involves possibly working with the Github API, page creation and routing in Gatsby. This seems like it would be a good PR, the only issue is that I am unsure of the activity of the project. I really like working on openclimatefix, because it's mission is something I agree with and the Github community has been very helpful to me. What started out as an emergency pull request at the end of Release 0.2 has become a project I genuinely care about. But if I hear nothing back from them I have to find something else.

Release 0.3

I have contacted Flo, the main contributor I have interacted with on the repository, to see if my solution can be merged. If not, I will work until it is. It seems like Flo is a bit distracted, and not really working on the website, so for now I will just have to wait for a response. 

Thursday, November 21, 2019

Release 0.4 - More Github, More Problems

And so it continues....

After the spectacular disaster that was Release 0.3, I have returned this week for my fourth and final release. At this point, I am happy with everything I have done so far in this course, and these last few weeks will test everything I have learned so far. To start, I am going to go over some updates from last week's posts, and explain my plans for Release 0.4.

Release 0.3 Updates

I received some change requests from Flo as follows:
  • Make different grid sizes for different screen sizes
  • Try to use tailwind classes to attempt this
  • Change some syntax and naming
  • Adjust font size and justification
The conversation is listed here. So I successfully made these changes to the code aside from using tailwind. I couldn't get tailwind to work with the .module.css file type, as tailwind works better with regular css. I used some of the syntax to adjust font size and justification, but I couldn't the other way. I ended up using the @media method to detect screen size and adjust the grid to the required size. Some of the changes made also help with a compression issue, where some images were being squeezed to fit in the grid at certain screen sizes.

@media method

300px is for phone screens, 768px is for tablets, 992px is for smaller computer screens, and 1200px is for larger screens. Each of these has a different number of columns to fit the screen. 

Telescope - Release 0.4

I have picked an issue to work on for Release 0.4, to continue my work from 0.3 on documentation. My plan is to work on this issue to start, and then continue to monitor and update documentation as the Release continues. Part of my problem is that I have no way as far as I know to install docker in my current environment. I use WSL (Windows Subsystem for Linux), which cannot use docker, and I need Windows Professional to use docker desktop. I am stuck working on documentation and organization until I can get docker up and running. I am going to continue researching ways to get docker going over the weekend.

External PR - Release 0.4

I really enjoyed working on open climate fix this week, it made last week's madness feel a lot more legitimate. In terms of an external PR for Release 0.4, I really want to work on a page for a website. I would love to continue working on open climate fix, however there doesn't seem to be an escalated issue on that site at the moment, except for a contributing page which I am unsure of the complexity of. I am currently searching for an issue similar to what I have worked on these past 3 months. My criteria for a new issue are the following:
  • UI related (components, new pages, new features)
  • In an environment I am somewhat familiar with (Gatsby, React, Angular, Javascript, etc)
  • An escalation from my previous issue (a gatsby component with simple file reading)
Some issues that I have found that somewhat satisfy my criteria:

Friday, November 15, 2019

Release 0.3 - Third time's the charm?

OpenClimateFix - A New Issue

Here we are again. I have bounced from issue to issue, hitting roadblocks as I go. I know that the issue I have picked here is not as complex as tania-core, but I believe it is a little more complex than WritingSwarm and is in a system I am familiar with. I am devoting all of my available time to completing this issue, and I intend to have a pull request up by tomorrow night. In order to do that, I will have to work hard in order to get this done, and I am committed to doing that.

The Issue

The issue I have chosen is to create a contributors showcase component, for a new Get Involved page on the openclimatefix website. I have worked on openclimatefix before, as my last pull request for Release 0.2. In order to populate the contributors showcase, I must read a json file containing each contributor and display their name, profile url and github avatar.

Setting up the Environment

I already had the environment set up as I have worked on this project before. It uses yarn to install dependencies, Gatsby as a framework for development, and a host of dependencies to run tests and ensure consistent formatting. I updated my fork of the repo and created a new branch to work in. 

In terms of research, although I have used Gatsby before, I have never created a new component. I had to figure out the conventions used in the openclimatefix site and copy those conventions. I also had to figure out how to read a static json file in Gatsby. Turns out you need an extra plugin, called gatsby-transformer-json in order to read in json files using graphql. graphql comes with Gatsby and is used to manage static and dynamic data in a website. I also had to learn what a StaticQuery is in Gatsby, in order to display the results of the graphql query. 

The Solution and Pull Request

The Component Code

Component Display Code
This function returns the Contributors header with each contributor displayed below it. The StaticQuery has 2 properties; query and render. query uses graphql to query the json file named contributors.json and get the contributors attribute's contents, in order to create a javascript object containing each contributor's name, profile (url), and avatar (image url). The render property will run a function to render the data generated using the function getContributors().

getContributors()

getContributors() receives the object generated by the graphql query
getContributors() receives the data from the json file and for each contributor makes a <div> element to contain the contributors profile image, and their name, which is also a link to their profile. In order for the render to work, a key must be chosen, so I used the name property for this key. For each contributor in the contributors array, a div with their information will be appended to the cont array, and return to the component code. This will populate that code with each grid item, ready to be displayed.

CSS

Basic CSS to set up the grid

Css works a little differently in Gatsby than it does in regular HTML5 development. It is called like an object, with each class represented and called to as an attribute of that object. Whenever you see "styles.something" in the className properties above, they link to a class in the CSS document.

Final Result


Interactions with Flo (FWirtz)

Flo is the maintainer of the repository, and is very helpful and communicative. She and I have not talked much during this process, but she did assign me to the issue. We have talked before on my previous 2 issues in openclimatefix,  and I expect to hear from her about this pull request, although I don't think she has reviewed it yet.

Release 0.3 - Overall Experience

I felt very rushed in this release. We only had 2 weeks really, considering Release 0.2 was due Oct. 31st and I needed to work on it until then. I had other commitments, and as hectic as October was, November has been worse so far. There are only about 4 weeks left in the semester, and in that time I have to complete Release 0.4. Although Release 0.3 was difficult for me, and I'm not sure how well it went, I know that I'm much wiser and more aware of my abilities. For one thing, I am going to avoid databases for Release 0.3. Secondly, I am going to find a good issue early on, and get started on it right away. 

Thursday, November 14, 2019

Release 0.3 - Failure, Again...

The old song and dance...

So we've come to another deadline, and again I am running late. I had an issue to work on, with tania-core and it's SQL to CSV conversion. Turns out this was a much larger undertaking than I expected. I might still work on it for Release 0.4, but I'm unsure if I have the knowledge in SQL and Vue to even attempt tania-box. I worked on it this weekend but with little success, and with time running out and pressures from other courses, I had to find another issue quick. To my surprise, I rediscovered an issue I had commented on but had never been assigned to and jumped on it. I am working on WritingSwarm now, which has already been an interesting project and has opened my mind to a completely different world.

WritingSwarm and the Issue

WritingSwarm is a Dapp for writing short stories collaboratively using the Etheresphere Swarm backend. Sounds complicated, and it is, but first what is a Dapp. A Dapp is a distributed app, meaning that it is run on a distributed system. A distributed system is a a network of computers that pass information between them to achieve a common goal. Swarm is a peer-to-peer distributed storage platform and content distribution service. WritingSwarm uses swarm to develop and share short stories. 

The issue involves adding an auto-save feature to the WritingSwarm site, which is built in Angular. I have used Angular before but never Swarm, so it has been an interesting experience to say the least. Basically, the site already has a save feature, but the user is required to press a save button in order to save their work. My task is to add a toggle switch that toggles between "ON" and "OFF" states for the WritingSwarm environment, and in order to do that I need to use Angular Material, which I have not used before.

The Environment

The hardest part of this issue is getting Swarm up and running, because connecting to it is not simple. In order to develop this project, I have to set up a Swarm node, learn Angular Material, and adapt the existing code to provide the auto-save feature. 

In order to get swarm, I had to use go, which I was fortunate enough to have installed when I was trying to get tania-core up and running. I also had to get geth, which stands for go-ethereum, the official go implementation of Ethereum. Ethereum is basically an open source platform for Dapps. So in order to run WritingSwarm, I had to get an account in Ethereum for go, and download swarm to set up my node. Once I logged in, my node was set up and I was able to serve WritingSwarm on my localhost.

So it started running, however when I try to add a file, I keep getting an error message. I sent a message to Kortatu, the owner of the repository and I am waiting to hear back.

UPDATE: Nov 14

I have not heard back from Kortatu, and the more I hear about Swarm the more I dislike using it. I don't really trust it, and was only using it because I was running out of time. Well now I'm stuck and out of time, so my only option is to find something else. It is very late and I don't expect to do well on this, but I need something to hand in on Friday. I have found a new issue on an old project and will be detailing it in another blog post. I hope to have something working by Friday, and for Release 0.4 I will be more proactive on my external pull request, like I was on my internal one.

Thursday, November 7, 2019

Release 0.3 - A new experience altogether

Release 0.3 - Telescope and Tania-Box

What is Release 0.3

As my open source course moves on from Hacktoberfest, we have switched to working internally and externally on open source projects. For release 0.3 I had to work on telescope, a new app for tracking the blog posts of all the students in my course, and I had to find another issue to work on and submit a pull request. I have had a lot of success in one of these, and little in the other. I will provide an update next week on how things have progressed.

Telescope

I have made a lot of progress in Telescope. I have already had my issue read, created my solution, and had my pull request merged after a review by two of my peers. Let's start with the issue.

The Issue

My issue was to create a CONTRIBUTING.md to create a centralized place for environment setup instructions, prerequisites, and testing information. It would also include a list of the technologies we are using, as well as provide a guide on making issues and pull requests. Finally, it would display the MVP features, features we need to create the most basic version of the website.

Research

I looked at many different CONTRIBUTING docs across many projects in order to get an idea of what I wanted to put in there. I also needed to research Redis, as at the time it had just been added as a requirement to the project and the instructions for set up were non-existent. I researched for a few hours and got all the information I needed. Redis is an open source noSQL database. It is used by our app to create a queue to load blog posts from the list of feeds in the feeds.txt file.

The Solution and Pull Request

I created the CONTRIBUTING.md document, which I will link to here. This document contains information on creating issues, and creating pull requests, based on course requirements during Hacktoberfest. It also provides links to the prerequisites for cloning and running the code. These prerequisites are Node.js and Redis. Instructions are also provided for installing Redis on Linux and on WSL (Windows Subsystem for Linux). I did not include the Windows or MacOS set up instructions, as I cannot test either. I do not use Windows for my app development environment, I use WSL. I do not own a Mac, nor do I have the means to get a virtual machine version. I left those to others who need issues to work on. 

I also included a checklist version of the MVP Features list, to provide a way of checking whether or not we have completed any or all of the goals laid out there. Finally, I included the setup instructions to get the code running. 

Setup Instructions

You need to run "npm install" to install dependencies, and "npm start" to start running the code. If you get error messages, you may have to run "redis-server" in a separate window in order to start the server, and then try again.

So now what?

Well, Release 0.3 is part internal and part external. Although the mandatory internal part is technically done, I still have to work on an external issue. And the one I've found looks quite difficult. 

Tania-Box

I actually asked and was assigned to this issue about 2 weeks ago. It is more complex than any issue I have faced so far, and I am even considering moving this issue to Release 0.4. It depends on how complex it is going to get. Right now, I have set up the environment on my home PC, and I am starting to understand how it works. Progress is not great, as I have too much in other courses going on this week. I am hopeful that I'll have time this weekend to work on it.

The Issue

The issue is to create a new menu for converting the SQL database to a CSV file. I am unfamiliar with the database, and how I would even go about doing this. I am going to attempt it this weekend, and if I cannot, I will find something else to do.

Background Research

I have done some research on it, as it uses Go primarily, a language I am fairly unfamiliar with. It also uses SQLite, which is a database engine, and working with databases was one of my goals from Hacktoberfest.