Long time no write stories.. A lot of stories i wanna tell about my journey in Machine Learning in past 2 years.
And i wanna start with the Machine i created for Recommender System in Vidio recently (the company i work at). Ys-Mean Recommender System. The very simple Hybrid Recommender System.
This i created by my self, my idea, you won’t found this at internet or papers. But i can guaranty, this Technic is very easy to understanding and effective for Recommender System.
Benefit
Fill each other’s gaps
Every each of Recommender System has some minus parts or we can say cons. such as:
Collaborative Filtering cannot cover cold start, Collaborative Filtering only see user behavior (sometimes when we have not enough user, it cannot be applied to our Recommender System).
Content Based just looking at the metadata of the contents that you want to recommend. And the metadata mostly provided by human, actually it can be provided by machine. When the human or the machine provides wrong metadata then it will gives wrong recommendation. Also this method doesn’t see user behavior.
And when we combine those Recommender Systems into 1 Recommender System, it will fill each other’s gaps.
Can control or maintain the proportional
With Using simple combine using mean or average, we can control the proportional of model. Let’s say we want to give more value on Content based score, then simply we can make double of content based score then average them.
Explanation
Softmax
First of all you must have a good understanding about Softmax. Basically Softmax is one of Activation Function. Here a story that i created about activation function including Softmax.
“get confused about Softmax! and after that, get a good understanding about Softmax is!”. Because this is the basic that helping us a lot in building Machine Learning Model.
Softmax is the main player in this Ys-Mean Recommender System. Before you calculate the average or mean of ‘Y’s you have to make sure the Y’s are the result that come from softmax activation function.
Use Softmax for Recommender System? Seriously?
Yes, of course i am serious!. y not? ups.. why not?.
Usually we use softmax for categorical Model, like to define it is a cat, cow, bunny or dog. But how can i use softmax for recommender system?
Softmax gives us probability values, for example (0.3, 0.1, 0.4, 0.2). If we sum the values it will be 1. the values represents of scores of each categories.
Let’s say we have a model that has softmax activation function at the last layer and also we have this picture. “oh God,, this is so cute! ♡”
Let’s say we predict this picture using the model and the model gives us some scores. Scores of this picture are: the cat is 0.3, cow score is 0.1, bunny is 0.4, and dog is 0.2. Then we can say it is a Bunny. Aaaaaand based on the scores we can say it is more like cat than dog and it is more like dog than cow. And of course it is not more like cat than bunny.
Okay, that’s not about recommendation right? oh.. you are wrong.. from the model result scores, i can recommend to you that the picture is the picture of a bunny, if you don’t like that this is a bunny, then i recommend to you that this is a cat.
Seee,,, we can say the model gives us the recommend of name or kind of animal based on the picture.
Now we can say, the model is just like Recommender System Model. Then we also can say we can use softmax for Recommender System.
Recommender System with Softmax
i will not tell the detail about how to create recommender system with softmax in this story, at least not this time. but i promise i will create another story that telling you about how to create recommender system with softmax. the stories will include Collaborative Filtering Recommender System with softmax, Content Based Recommender System with softmax and also Knowledge Based Recommender System with softmax.
This is an very simple example code, 1 of Recommender System i created (collaborative filtering/user behavior), using softmax activation function:
model = Sequential()
model.add(Embedding(classes_size, 1024, input_length=5))
model.add(Flatten())
model.add(Dense(1024, activation="relu"))
model.add(Dense(1024, activation="relu"))
model.add(Dense(classes_size, activation='softmax'))Wider Deep Learning
If we say deep learning, it has a depth. If we say wider deep learning, it has depth and also width.
At this picture you see not only deep but also wide.
Simple, you just make sure every Network (collaborative, content based, knowledge based) using softmax for the last activation function And also return the same format, same format what i mean is the Y scores every each Network represents the same list of categories, in recommendation case is list of contents that you want to recommend.
And at the end, combine the result scores by calculate the average. why calculate the average? because we want to maintain the sum of all of the scores not higher than 1. To make it fast when we want to proceed then present to the users.
The Result
I used Tensorflow whenever created every single Recommender System model in Vidio. Also the Ys-Mean Recommender System i use Tensorflow
The model summary just like this picture.
To achieve Ys-Mean Recommender System i created some custom layer, including the last layer.
The last layer just like this:
class YsMeanLayer(tf.keras.layers.Layer):
def __init__(self):
super(YsMeanLayer, self).__init__()def call(self, inputs):
return tf.reduce_mean(inputs, axis=0)
i use simply reduce_mean because i don’t want to changed the proportional. But if you want to change the proportional you can doubled the scores that you want to give more value.
Finally
I hope this story gives you more value, and helping you to create Recommender System.
This is Ys-Mean,, and you can called it “Wise Man” :P
