GSOC 2016- Making Label Detection results configurable and improving documentation- Week 12

TL;DR Last week I had worked moving the helper functions for filling Alt Text of image file to a new service; and moving the reused/supporting functions of the tests to an abstract parent class, GoogleVisionTestBase. This week I have worked on improving the documentation of the module and making the label detection results configurable.

With all major issues and features committed to the module, this week I worked on few minor issues, including the documentation and cleanup in the project..

It is an immense pleasure for me that I am getting the feedbacks from the community on the Google Vision API module. An issue Improve documentation for helper functions was created to develop more on documentation and provide the minute details on the code. I have worked on it, and added more documentation to the helper functions so that they can be understood better.

In addition, a need was felt to let the number of results obtained from the Vision API for each of the feature as configurable, and allow the end user to take the control on that. The corresponding issue is Make max results for Label Detection configurable. In my humble opinion, most of the feature implementations and requests to the Google Cloud Vision API have nothing to do with allowing the end user to configure the number of results. For instance, the Safe Search Detection feature detects and avoids the explicit contents to be uploaded, and does not need the number of results to be configurable. However, the taxonomy tagging using Label Detection should be user dependent, and hence, I worked on the issue to make the value configurable only for Label Detection purpose. This value can be configured from the Google Vision settings page, where we set the API key. I have also developed simple web tests to verify that the value is configurable. Presently, the issue is under review.

I have also worked on standard coding fixes and pa-reviews and assisted my mentor, Naveen Valecha to develop interfaces for the services. I assisted him on access rights of the functions, and fixing the documentation issues which clashed with the present one.

Lastly, I worked on improving the README and the module page to include all the new information and instructions implemented during the Google Summer of Code phase.

With all these works done, and all the minor issues resolved, I believe that the module is ready for usage with all the features and end user cases implemented.

Next Week, I’ll work on creating a video demonstration on how to use Google Vision API to fill the Alt Text attribute of an image file, detect the emotion in the user profile pictures and to group the similar images which share the same dominant color.

GSOC 2016- Moving supporting functions to services and abstract parent classes- Week 11

TL;DR Last week I had worked on modifying the tests for “Fill Alt Text”, “Emotion Detection” and “Image Properties” features of the Google Vision API module. The only tasks left are moving the supporting functions to a separate service, in addition to, creating an abstract parent class for tests and moving the functions there.

The issues Alt Text field gets properly filled using various detection features, Emotion Detection(Face Detection) feature and Implementation of Image Properties feature of the Google Vision API module are still under review by my mentors. Meanwhile, my mentors asked me to move the supporting functions of the “Fill Alt Text” issue to a separate service and use it from there. In addition, they also suggested me to create an abstract parent class for the Google Vision simple tests, and move the supporting functions to the parent class. Thus, this week, I contributed to follow these suggestions and implement them out.

There are few supporting functions, namely, google_vision_set_alt_text() and google_vision_edit_alt_text() to fill the Alt Text in accordance to the feature requested from the Vision API, and also to manipulate the value, if needed. I moved these functions to a separate service, namely, FillAltText, and have altered the code to use the functions from there instead of directly accessing them.

In addition, there are a number of supporting functions used in the simple web tests of the module, to create users, contents and fields, which were placed in the test file itself, which in one way, is a kind of redundancy. Hence, I moved all these supporting functions to abstract parent class named GoogleVisionTestBase, and altered the test classes to extend the parent class instead and in place of WebTestBase. This removed the redundant code, as well as, gave a proper structure and orientation to the web tests.

These minor changes would be committed to the module directly, once the major issues are reviewed by my mentors and committed to the module.

GSOC 2016- Modifying tests for “Fill Alt Text”, “Emotion Detection” and “Image Properties” features of Google Vision module- Week 10

TL;DR Last week, I had worked on and developed tests to ensure that the Alt Text field of an image file gets filled in accordance to the various detection features of the Vision API, namely Label Detection, Landmark Detection, Logo Detection and Optical Character Detection. This week I have worked to modify and add tests to various features of the Google Vision module, namely filling of Alt Text field, emotion detection of user pictures and grouping the image files on the basis of their dominant color component.

My mentors reviewed the code and the tests which I had put for review to get them committed to the Google Vision API module. However, the code needs some amendment pointed out by my mentors, which was to be corrected before commit. Hence, I spent this week working on the issues and resolving the flaws, rather than starting with a new feature.

Let me start discussing my work in detail.

I had submitted the code and the tests which ensure that the Alt Text field gets properly filled using various detection features according to the end user choice. However, as was pointed out by my mentor, it had one drawback- the user would not be able to manipulate or change the value of the field if he wishes to. Amidst the different options available to the end user to fill the alt text field of the image file, there was a small bug- once an option is selected, it was possible to switch between the options, however, disabling it was not working. After, been pointed out, I worked on modifying the feature and introducing the end user ability to manipulate the value of the field as and when required. Also, I worked on the second bug, and resolved the issues of disabling the feature.

Regarding the Emotion Detection(Face Detection) feature of the Vision API, I was guided to use injections instead of using the static methods directly, and to modify variables. For example, the use of get(‘entity_type.manager’) over the static call \Drupal::entityTypeManager(). Apart from these minor changes, a major issue was the feature was being called whenever an image file is associated with. However, I need to direct it to focus only when the user uploads an image, and not on its removal (as both the actions involves an image file, hence the bug).

In the issue, Implementation of Image Properties feature in the Vision API, I had queried multiple times to the database in the cycle to fetch results and build the routed page using the controllers. However, my mentor instructed me that its really a bad way of implementing the database queries to fetch the results. Hence, I modified the code and changed them to single queries to fetch the result and use them to build the page. In addition, I was asked to build the list using ‘item_list’ instead of using the conventional ‘#prefix’ and ‘#suffix’ to generate the list. Another important change in my approach towards my code was the use of db_query(), the use of which is deprecated. Hence, I switched to use addExpressions() instead of db_query().

Presently, the code is under review by the mentors. I will work further on them, once they get reviewed and I get further instructions on it.

GSOC 2016- Developing tests for “Fil Alt Text” feature of Google Vision module- Week 9

TL;DR Last week, I had worked on and developed tests to ensure that the similar images are grouped in accordance to the Image Properties feature of the Vision API. The code is under review by the mentors, and I would continue on it once the review is done. Meanwhile, they also reviewed the “Fill Alt Text” feature issue, and approved it is good to go. This week, I have worked on developing tests for this issue.

An important feature that I have implemented in the Google Vision API module is the filling of Alt Text field of an image file entity by any of the four choices- Label Detection, Landmark Detection, Logo Detection and Optical Character Detection. My mentor suggested me to check the availability of the response and then fill the field, as we can not fully rely on the third party responses. With this minor suggestion being implemented, now its time to develop tests to ensure the functionality of this feature.

I started developing simple web tests for this feature, to ensure that the Alt Text field is properly filled in accordance to the choice of the user. It requires the selection of the four choices one by one and verify that the field is filled correctly. Thus we require four tests to test the entire functionality. I have added an extra test to ensure that if none of the options are selected then the field remains empty.

I created the image files using the images available in the simpletests. The images can be accessed through drupalGetTestFiles(). The filling, however, requires call to the Google Cloud Vision API, thus inducing dependency on the API key. To remove the dependency, I mocked the function in the test module, returning the custom data to implement the feature.

The first test ensures that the Label Detection feature returns correct response and the Alt Text field is filled correctly. The simpletest provides a list of assertions to verify it, however, I found assertFieldByName() to be most suitable for the purpose. It asserts the value of a field based on the field name. The second test ensures that the Landmark Detection feature works correctly. Similarly, the third and fourth test ensures the correct functionality of the Logo and the Optical Character Detection feature.

The fifth test which I have included perform tests when none of the options are selected. It ensures that under this case, the Alt Text field remains empty, and does not contain any unwanted values.

I have posted the patch covering the suggestions and tests on the issue queue Fill the Alt Text of the Image File using Google Vision API to be reviewed by my mentors. Once they review it, I would work on it further, if required.

GSOC 2016- Developing tests for Image Properties feature of Google Vision module- Week 8

TL;DR In the past two weeks I had worked on using the Image Properties feature offered by the Google Cloud Vision API to group the image files together on the basis of the dominant color components filling them. In addition, I had also worked on detecting the image files and filling the Alternate Text field based on the results of Label/Landmark/Logo/Optical Character Detection, based on the demand of the end user. This week, I have worked on and developed tests to ensure that the similar images are grouped in accordance to the Image Properties feature of the Vision API.

At present, the Google Vision API module supports the Label Detection feature to be used as taxonomy terms, the Safe Search Detection feature to avoid displaying any explicit contents or violence and the User Emotion detection to detect the emotions of the users in their profile pictures and notify them about it.

I had worked on grouping the images on the basis of the dominant color component(Red, Green or Blue) which they are comprised of. I got the code reviewed by my mentors, and they approved it with minor suggestions on injecting the constructors wherever possible.

Following their suggestions, I injected the Connection object instead of accessing the database via \Drupal::database().

After making changes as per the suggestions, I started developing simple web tests for this feature, to ensure that the similar images gets displayed under the SImilarContents tab. It requires the creation of new taxonomy vocabulary and adding an entity reference field to the image file entity. After the creation of the new Vocabulary and addition of the new field to the image file, I created the image files using the images available in the simpletests. The images can be accessed through drupalGetTestFiles(). The first test ensures that if the Vocabulary named ‘Dominant Color’ is selected, the similar images gets displayed under the file/{file_id}/similarcontent link.

The grouping, however, requires call to the Google Cloud Vision API, thus inducing dependency on the API key. To remove the dependency, I mocked the function in the test module, returning the custom data to implement the grouping.

To cover the negative aspect, i.e. the case when the Dominant Color option is not selected, I have developed another test which creates a demo vocabulary to simply store the labels, instead of the dominant color component. In this case, the file/{file_id}/similarcontent link displays the message “No items found”.

I have posted the patch covering the suggestions and tests on the issue queue to be reviewed by my mentors. Once they review it, I would work on it further, if required.

GSOC 2016- Detection of image files and filling its Alt Text field- Week 7

TL;DR Previous week I had worked on detecting the emotion in the profile pictures of the users, and notifying them to change the image if they do not look happy. The work is under review by the mentors. Once it gets reviewed, I would resume it if it needs any changes. This week I have worked on filling the ‘Alt Text’ field of an image file based on any one of the method selected by the end user- Label Detection, Landmark Detection, Logo Detection and Optical Character Detection.

Last week, I had worked on implementing the Face Detection feature in the Google Vision API module. The code is currently under the review by the mentors. Once, they review it, I would develop further on it if it requires any changes.

The Google Cloud Vision API provides the features to detect popular landmarks in an image(Landmark Detection), logos of popular brands(Logo Detection), texts within an image(Optical Character Detection), in addition to Label Detection. These features, though of less significance, are helpful in identifying an image. Hence, I have started working on implementing a new helpful case for the users- Filling of the Alternate Text field of an image file using these features.

The Alt Text field of the image file entity is modified to incorporate the options to fill the field using the features. The user may select any one of the four options to fill the Alt Text field of the image.

Coming to the technical aspect, I have made use of hook_form_BASE_FORM_ID_alter() to alter the Alternate Text field of the image file entity. I have modified the edit form of the Alt Text field to add four radio options, namely- Label Detection, Landmark Detection, Logo Detection and Optical Character Detection. The user may select any of the options and save the configuration. The Alternate Text field would be filled up accordingly.

Presently, the code is under the review by the mentors. Once it gets reviewed, I would make suggested changes, if required.

GSOC 2016- Starting with Face Detection feature of Google Cloud Vision API- Week 6

TL;DR Previous week I had worked on grouping the contents based on the dominant color component in their images, if present. The work is under review of the mentors. And once, it gets reviewed, I would work further on that issue. Meanwhile, I have started developing and implementing the Emotion Detection feature of the Google Cloud Vision API. It would detect the emotion of the person in the profile picture uploaded, and if the person looks angry or unhappy, he would be notified thereof. This feature is especially important when building sites for professional purposes, as the facial looks matters a lot in such cases.

Last week, I had worked on implementing the Dominant Color Detection feature in the Google Vision API module. The code is currently under the review by the mentors. Once, they review it, I would develop further on it if it requires any changes.

Hence, meanwhile, I have started working on implementing a new feature Face Detection in an image. This feature gives us the location of the face in an image, and in addition, the emotions and expressions on the face.

I have used this feature to detect the emotion of the person in the profile picture uploaded by him/her. If the person does not seem happy in the image, he/she is notified thereof of their expressions. This is especially useful when the end users are developing a site for professional purposes, as in professional matters, expressions matters a lot.

Coming to the technical aspect, I have made use of hook_entity_bundle_field_info_alter() to alter the image fields, and check the emotions in the uploaded images. This function has been used, as we only want to implement this feature on the image fields. If the image is not a happy one, then appropriate message is displayed using drupal_set_message(). This feature also makes use of Constraints and Validators just like the Safe Search detection feature.

Presently, the code is under the review by the mentors.

In addition to the implementation of Face Detection, I also worked on expanding the tests of the Safe Search Detection feature of the Google Vision API module to test other entities as well, in addition to the nodes. I have expanded the tests to test the safe search constraint on the comment entity as well.

This requires the creation of a dummy comment type, adding an image field to the comment type, and attaching the comment to the content type.

The image field contains the safe search as the constraint on it. This test is basically similar to the tests present in the module for the node entity.

The code is under review by the mentors and would soon be committed to the module.

For reference on how to create dummy comment types and attaching it to the content types, the CommentTestBase class is very helpful.