完善一个叫Scalica的项目，实现要求的新功能，最终需要部署到Google Cloud Engine上。
Goal
The final result we want to achieve is to extend the function of Scalica so
that it can automatically match two users for a dinner date.
In general the project could be divided into three subsections:
- Requesting and storing user input
- Analyzing user input and generating a matching rank
- Presenting matching information to user
Firstly, we will want to augment the functionality in Scalica. We would like
to ask the users some questions. There are 20 rating questions such as “Do you
like spicy food?” “Do you like French food?” each question with a scale from 1
to 5 (1: dislike – 5: Like a lot). The service randomly chooses 3 questions,
and ask the user to indicate their preference. These data will be stored in a
table with userID as the primary key, and each column representing a question
storing the rating value. All these data are considered to be private data. So
we will also have to restrict the viewing or changing of them to unauthorized
users. For example, some pervert who is trying to setup a date for himself and
another person he fancies will not be able to do that.
A is responsible for this part (augmenting Scalica (and its data models) to
store additional info), and he would like to use either Java or Python and
Django for this part. More on database design: to make the database scalable,
we will use horizontal partitioning to partition the user info data. Even
though we only get one physical node for database, we will create some logical
shards so that it will be easy when the amount of data grow and we need to
redistribute them across different physical nodes. Each time a new entry is
created, its ID will contain an identifier marking the logical shard which
it’s in.
Secondly, we will want to use these data to match these people. We will be
using the latent factor model to achieve this. Normally, the latent factor
model will have a data table that store each user and their preference level
to multiple labels. In our ca se, this matrix A will be a matrix of users vs.
all the different labels created by users. These labels are the favourite
books, users’ jobs, their favourite movies and etc. In the actual matrix, each
value of the matrix element is the rating we got from first step corresponding
to (user, label). Since it is unrealistic for users to rate their preference
level for each of the label in the matrix, there will be many blanks in the
matrix. We will then compute the latent factor for each user to the other
labels to substitute these blanks. To compute the latent factors. We would
need to create two matrix out of the original matrix A. One would be the
matrix that represents users vs. flavors. The other one would be flavors vs.
labels. By doing a gradient descent on the product of these two matrices, we
would be able to get the latent factor matrix. We could then use the latent
factor matrix to get labels that were not rated by the user but now it is. For
the completed matrix, each row would be the interpreted preference array for
the user against the labels. We then find out for each user, whose preference
array is most alike. We would do this by computing the dot product of the
difference array between two users by itself. The users with lowest three such
values will be recommended to each other. Depending on the number of users we
have and the frequency they choose to update their preference information, we
will need to update the preference matrix in different windows of time. B will
be responsible for the second part. For the language used here. He is
considering either Python or C++. Former for the reason that it is easy to do
matrix arithmetic with Python. The later been that it might be faster to use
C++. All the above service will be run on a cloud server from Google’s Cloud
Engine. We will be using Redis to store the list of suggestions for it is
better supported than memcached.
Thirdly, we will have two different database, one is for the user information
and the other is for the result of survey. Both of them will be horizontal
partitioned and logical sharded for the expansion in the future. The host of
matching algorithm will analyze the data from the survey database. At the end
of every finished calculation, it will update all changes to the suggestion
list in Redis. The algorithm will run every time when there is a change in
survey database. C is responsible to construct survey database.
Optional: We will need to be presenting these informations to the users. We
will need to guarantee that both user’s personal information is not leaked in
anyway to the other prior to their meeting. First, we will present the
suggested restaurant, the price range, and the dining time to both users in
the match. If both users agree to meet, the system will provide them more
detailed and personal information, such as interests, favorite food and etc.
If one of the user in the match decides to not meet, the system will hide all
the information presented before from both users and suggest a new match to
both users according to the next higher rank in the system. The third part
require techniques in visualizing the data. After achieving basic function
mentioned above, another main focus is beautifying the user interface and the
data visualization. C is responsible for this part, and he would like to use
python and Django for this part. The packages I might use are Bokeh for
visualization with Python.
