We made it! (aka Outreachy wrap-up report)

I can hardly believe it – the project is complete and Outreachy is nearly over! When I started, I was plagued with doubts about whether or not I could do it – I didn’t have a clue on how to do many of the things that we’d planned. But it all worked out in the end, and in fact turned out better than I expected. Perhaps it’s okay to not know how you’re going to do something, perhaps it’s a normal part of a developer’s life? That’s one of the more important lessons I’ll be taking away from this, at least.

But enough of my rambling, here’s the final summary of the Upload to Commons Android app project. The main goal of the project was to improve the image categorization functionality of the app by offering relevant category suggestions based on geolocation, and making category search more flexible. Download the latest version of the app here, or check out its source code here!

Category suggestions based on geolocation

If a picture with geolocation is uploaded, nearby category suggestions are offered based on the categories of other nearby Commons images. We query the Commons MediaWiki API for the Commons categories of pictures with geolocation within a radius of 10km of the uploaded pic’s geolocation. The categories found are also cached using the quadtree library, so that if another picture with similar geolocation is uploaded, the cached categories are displayed and the API is not queried. This is done to reduce the load on the API, as people often upload several pics with similar locations.

Example: Uploading an image of the Hamilton Gardens with geolocation:

Entering the title and description:

Category suggestions obtained:

If a picture with no geolocation is uploaded, nearby category suggestions can be offered based on the user’s current location. This is useful if someone doesn’t want to geotag their pictures, and uploads them while still in the same area. The feature is optional and only works if enabled in Settings (disabled by default for privacy).

Enabling the ‘use current location’ settings:

More flexible category search

Category search (when typing in the search field) has been made more flexible. Prior to this, category search was done solely by prefix search. E.g. if you searched for ‘latte’, you would only get categories that start with ‘latte’, which excludes many potentially relevant categories (i.e. ‘iced latte’). Prefix search is useful when you are certain of what the exact name of the category you want is, but not so useful if you are unsure. So we wanted to retain prefix search, but also provide additional suggestions for relevant categories that do not share that prefix.

We query the Commons MediaWiki API for pages of type “Category” (srnamespace=14), maximum 10 results (srlimit=10). These results are aggregated with the prefix search results and the duplicate suggestions are eliminated before they are displayed.

Example: Searching for ‘latte’ categories:

Advertisements
We made it! (aka Outreachy wrap-up report)

Outreachy Week #10 (and message to potential new interns, if any of you read this)

Got the optional task up early this week! Good thing, too, as I haven’t been feeling very well since Tuesday. 😦

So that’s pretty much all the MVPs that have been scheduled in my proposal, woohoo! 🙂 Not sure what I should be doing after this – probably do another announcement on the mailing lists, fix any further bugs that are reported, and start on the documentation and wrap-up report? We’ll see what the mentors think during our next meeting. I can’t believe the project is nearly over! It’s been a whirlwind and a half, but so very fulfilling. And I’ve learnt so, so much. Not just about coding, but about collaborating and FOSS as well. Good stuff.

Which reminds me – the application process for the next round of Outreachy and GSoC is starting soon! If any of you potential new interns are reading this – go for it! Really. Find an organization, a project/mentor, make your small contributions, and submit your application. Start as early as you can, but don’t worry too much about it, the folks in charge are very helpful and encouraging, and they most certainly do not bite. 🙂

If you are interested in applying to Wikimedia, this is an excellent place to start.

Outreachy Week #10 (and message to potential new interns, if any of you read this)

Outreachy Week #9

The bulk of the work planned for my Outreachy programme has been implemented – I can hardly believe it! Wrapped up the minimum viable product of Phase 2 at the end of last week, while fixing a stray bug from Phase 1.

I was actually asked what I’d like to do for this week, so I chose to work on the optional task in my schedule, which involves retrieving the user’s location if the uploaded image has no GPS coordinates, in order to offer suggestions for nearby categories. This would hopefully allow the feature added in Phase 1 (suggesting nearby categories based on an image’s coordinates) to be useful to more people, since it’s quite conceivable that someone might have location tagging disabled on their phone camera but might upload their pictures while still around the same area. Of course, there would be a preference to enable/disable this extra feature.

I wrongfully assumed that obtaining the user’s location would be as simple as extracting the GPS coordinates from an image, which I quickly found out was not the case. There are several ways in which it can be implemented, usually involving some form of LocationListener attached to a LocationManager. Mentor advised against using the fused location provider that required the use of Google Play services, so I’m trying out the method described at this blog and that for now. Ah, we learn new things every day…

On a side note, I’ve really enjoyed my time with Wikimedia Foundation so far, contributing to FOSS. Also, watching the number of active app installs on Google Play grow has been crazy addictive! I find myself wishing that it didn’t have to end… and maybe it doesn’t? I’ve been enquiring about further opportunities, and there appears to be a route I can pursue that might allow me to continue work on this app. We’ll see how it works out… 🙂

Outreachy Week #9

Outreachy Week #8

So, hey, there was actually a fairly simple solution to last week’s problem that I couldn’t see, as I was unaware of how anonymous inner classes worked in Java. Thank goodness for mentors who nudge you in the right direction!

The code in CategorizationFragment.java that was released in v1.8 of the app:


    private void requestSearchResults() {

        final CountDownLatch latch = new CountDownLatch(1);

        prefixUpdaterSub = new PrefixUpdater(this) {
            @Override
            protected ArrayList<String> doInBackground(Void... voids) {
                ArrayList<String> result = new ArrayList<String>();
                try {
                    result = super.doInBackground();
                    latch.await();
                }
                catch (InterruptedException e) {
                    Log.w(TAG, e);
                    //Thread.currentThread().interrupt();
                }
                return result;
            }

            @Override
            protected void onPostExecute(ArrayList<String> result) {
                super.onPostExecute(result);

                results.addAll(result);
                Log.d(TAG, "Prefix result: " + result);

                String filter = categoriesFilter.getText().toString();
                ArrayList<String> resultsList = new ArrayList<String>(results);
                categoriesCache.put(filter, resultsList);
                Log.d(TAG, "Final results List: " + resultsList);

                categoriesAdapter.notifyDataSetChanged();
                setCatsAfterAsync(resultsList, filter);
            }
        };

        methodAUpdaterSub = new MethodAUpdater(this) {
            @Override
            protected void onPostExecute(ArrayList<String> result) {
                results.clear();
                super.onPostExecute(result);

                results.addAll(result);
                Log.d(TAG, "Method A result: " + result);
                categoriesAdapter.notifyDataSetChanged();

                latch.countDown();
            }
        };
        Utils.executeAsyncTask(prefixUpdaterSub);
        Utils.executeAsyncTask(methodAUpdaterSub);
    }

Basically, I could separate most of the preexisting AsyncTask code into the MethodAUpdater.java and PrefixUpdater.java classes (that extend AsyncTask), and then override the methods that I need via anonymous inner classes. This allows me to add the results of the API queries to the ‘results’ member variable of the outer class, in order to aggregate the results in a LinkedHashSet before displaying them to the user.

I’ve released v1.8 to Google Play, here’s to hoping the flexible category suggestions work better for users! 🙂

Outreachy Week #8

Outreachy Week #7

Made it through Phase 1 and the mid-term evaluation! 🙂 We’re deep into Phase 2 now, which involves improving the as-you-type category search. The existing codebase uses a prefix search for these category suggestions, which means that if you don’t know exactly what your category name starts with, you won’t get any pertinent suggestions. E.g. searching for ‘latte’ when you really mean ‘iced latte’ will not get you anywhere.

With the API query that we plan to use, you absolutely will get ‘iced latte’ – as the second category suggested, in fact. 🙂 The aim is for us to run that query (dubbed Method A) in parallel alongside the existing prefix search, and display the Method A results above the prefix results.

This is easily the most complicated code I’ve worked on so far, I think, since it ties heavily into the existing structure of the as-you-type queries. Phase 1 did involve writing many lines of code, but my feature could be added in a more modular (that’s the right word I think?) fashion, without really needing to alter the existing code much. Phase 2 is quite different from Phase 1 in this regard.

Will update soon!

Outreachy Week #7

Outreachy Week #6

Our first big release is out! 😀 Version 1.6, which ACTUALLY displays nearby categories when a user uploads a photo, hit the app store this week. Ohohoho. *happy dance*

On a rather more serious note, I realized that launching a major update is not anywhere near as simple as I thought it would be. The first release last week (v1.5) was supposed to work, but in actual fact worked for almost nobody (which is why I didn’t count it). Because I stupidly hadn’t tested the app on my real device, assuming that it would work the same as on the emulator. But it did not – my URI to filepath converter was not robust enough to handle a wide variety of picture storage locations (local, external, sd card, etc) and did not account for changes in KitKat that mandated a different method. Fortunately, with the help of the aFileChooser open source library, we managed to get it working.

Today, thanks to a crash report uploaded by a Marshmallow user, I found out that the permissions system had changed in API 23 and runtime permissions are now required. Which means that we need to fix this ASAP, otherwise nobody on Marshmallow can upload anything. Ooof. (Edit: Fixed it! v1.7 released, hopefully that’ll sort it out…)

Still, I’m psyched to be reaching the end of Phase 1, and we’ll be moving on to Phase 2 (more flexible category suggestions) as soon as we can, while awaiting more feedback.

 

Outreachy Week #6

Outreachy Week #5

This week was a good one. 🙂 We found out that the MediaWiki API actually returns the n closest results when geosearch is used, so there is no need for the successive calls to the API that we’d planned – we just need to specify a large radius and the API will do the rest. Simplifies things greatly.

I also managed to get the caching of area -> categories done, using the varunpant/QuadTree library that my mentor suggested. Going to promote this library a little – it’s pretty awesome, even though there don’t appear to be many people using it. Lightweight and does what we need it to do, AND the code is very well documented! The latter is such a rarity in obscure GitHub projects that I heaved a huge sigh of relief when I saw this one. Made it so much easier to learn to use it. There was an issue with the lack of namespace when we were trying to create a JAR and import it, but that aside I loved this library.

Also pushed version 1.5 to Google Play – yay! Download it and let me know what you think, guys. Every additional install is an encouragement. 🙂 Edit: Whoops, don’t do that yet. Have an issue with app running on my real device… 😦

And….

I was raving about the joys of working remotely in my last post, but I also realized recently that it’s a double-edged sword for me. There isn’t really any predetermined ‘off’ time, and initially I didn’t enforce one. I kept pushing myself to do ‘just a little more’ every day, working more than 5 days every week, dealing with an email or post as soon as I saw it. To be clear this was entirely my own fault/issue, as neither my mentors nor org admins have ever complained about me ‘not responding quickly enough’ or ‘not doing enough work’. I just felt compelled to, and wasn’t allowing myself any proper downtime each day or week, except for the vacation I took for traveling early last week.

The net result is, I came close to burning out near the start of this week. I felt overwhelmed and exhausted and like I just couldn’t be arsed anymore. But then I realized it was my own doing – of course I’m burned out when I’m not allocating any proper time for relaxation. In fact the lack of downtime was probably making me less productive in the long run.

So what I’ve been trying to do is to enforce a downtime. I’ll work hard for 40 hours a week (not necessarily 9-5, I’ve never liked 9-5, but 40 cumulative hours), and the rest of the time I’m going to just allow myself to plug off and do something else. Hopefully that works out!

Outreachy Week #5