Discovery of Points of Interest with Different Granularities for Tour Recommendation Using a City Adaptive Clustering Framework

Increasing demand for personalized tours for tourists travel in an urban area motivates more attention to points of interest (POI) and tour recommendation services. Recently, the granularity of POI has been discussed to provide more detailed information for tour planning, which supports both inside and outside routes that would improve tourists' travel experience. Such tour recommendation systems require a predefined POI database with different granularities, but existing POI discovery methods do not consider the granularity of POI well and treat all POIs as the same scale. On the other hand, the parameters also need to be tuned for different cities, which is not a trivial process. To this end, we propose a city adaptive clustering framework for discovering POIs with different granularities in this article. Our proposed method takes advantage of two clustering algorithms and is adaptive to different cities due to automatic identification of suitable parameters for different datasets. Experiments on two real-world social image datasets reveal the effectiveness of our proposed framework. Finally, the discovered POIs with two levels of granularity are successfully applied on inner and outside tour planning.


Introduction
Nowadays, users like to share their travel experiences by uploading their photos to location-based social network (LBSN) services such as Flickr and Instagram. The relevant information of these photos (e.g., location, time and tags) makes the user-generated content (UGC) valuable for various data mining tasks. For instance, POIs are identified from geo-tagged photos (Crandall et al., 2009;Yang et al., 2011; and social image tags are analysed (Li et al., 2012;Zhang et al., 2017). This also makes LBSN services and data an important branch of social informatics (Fusco et al., 2010).
On the other hand, increasing demand for personalized tours for tourists in an urban area motivates more attention to POI and tour recommendation services. To recommend a ranked list of POIs or a sequence of POIs (i.e., a tour), these services or applications usually require a predefined POI database as the input data. According to preceding studies on POI identification or discovery (Crandall et al., 2009, Yang et al., 2011, the POI database can be constructed automatically by applying clustering algorithms over geotagged photos. Also, more POIs that are different from those on travel websites can be discovered from UGC. For example, Zhuang et al (2014) identify obscure sightseeing spots (i.e., less well-known while still worth visiting) by mining geo-tagged images. However, the granularity of POI is not considered well in the existing studies. All POIs, whether a scenic park or a small sculpture, are treated as equal, while tourists may want a more detailed recommendation. For example, Forbidden City, Beijing, China, and Central Park, New York City, USA are two famous POIs that cover large areas. It would be ambiguous to mark them as single POIs in the recommended tours because detailed inside routes are preferred by the tourists. Recently, Wang et al. (2018) proposed a twolayer tour recommendation framework to provide more detailed information about a recommended tour by introducing the concept of "Super-POIs". A Super-POI is a large-scale POI that covers several small-scale POIs, which should be explicitly indicated in the recommended tour rather than showing a unique POI. The left portion of Figure 1 shows an example of a tour generated by tour recommendation applications, and both locations 1 and 2 cover a large area. Tourists may like to know which places are more attractive to take photos inside location 1 and lead to different photo quantities, as shown in the right portion of Figure 1 (i.e., the heat map of photos taken in location 1). An inside route to guide tourists with detailed information is helpful when travelling to this type of places. However, constructing such a fine-grained POI database manually is very time-consuming due to a large amount of spots across the whole city. Wang et al. (2018) mark two Super-POIs and inside fine-grained POIs manually for their experiments, which is not enough for the scale of the entire city.
Existing studies on POI discovery do not consider the granularity of POI well, and cannot simultaneously support Supper-POI and fine-grained POI. Therefore, a method that discovers POIs with different granularities from geo-tagged images automatically is definitely desired by tour recommendation services. Also, parameter selection is another issue that should be addressed. Many POI discovery studies select parameters for their experiments subjectively. For example, Crandall et al. (2009) subjectively set the bandwidth parameter for mean shift clustering as 100 metres to identify landmarks. However, different cities obviously need different parameter settings, especially when considering the fine-grained POI. The spots inside Times Square and Central Park should be discovered with different parameter settings due to their different scales.
To this end, we introduce the task of detecting POIs with different granularities for tour recommendation applications automatically. We propose a clustering framework with a city adaptive parameter tuning process to discover POIs with different granularities for various cities. In order to verify our results on tour recommendation by applying the method of Wang et al. (2018), we consider two levels of granularity in this article: • Areas of interest (AOI): indicates the large-grained POI, such as a park or temple; is an area that attracts tourists to visit (refer to the locations 1 and 2 in the left portion of Figure 1). It can be represented as a polygon or minimum bounding rectangle (MBR) on the map. • Sightseeing spot: is a fine-grained POI inside the AOI, such as a sculpture or building inside a park, which is a specific coordinate that attracts tourists to take photos (refer to the high photo quantity spot in the right portion of Figure 1). It can be represented as a point or coordinate on the map.
More levels of granularity can be distinguished with our idea. After getting POIs with different granularity, both outside and inside travel routes can be generated by tour recommendation applications. We build on our conference paper (Sun et al., 2020) and the major contribution of this article can be summarized as follows: • We summarize the advantages and disadvantages of applying variant POI discovery techniques to detect sightseeing spots. • We propose a city adaptive clustering framework to discover POIs with different granularities.
The results reveal the effectiveness of our proposed method. • We apply the results to the tour recommendation application that verifies both outside and inside route planning is achievable.

Literature review
Discovering POIs from a collection of geo-tagged photos can be viewed as a clustering problem on twodimensional data (i.e., latitude and longitude). We first compare different clustering algorithms that have been applied to POI discovery and summarize the advantages and disadvantages of applying them to detect POIs with different granularities. Kennedy and Naaman (2008) applied the k-means algorithm to find landmarks from geotagged photos. The basic idea behind k-means is to separate a collection of locations into k clusters that make each coordinate belongs to the cluster with the nearest mean. The result of the k-means algorithm will assign a label to each location, and a mean coordinate is assigned to each cluster that can be treated as the detected POI. However, the k-means algorithm requires a parameter k to determine the number of clusters, which is very difficult to decide in advance. Especially in the task of sightseeing spot discovery, the number of sightseeing spots is much larger than the number of POIs.

Mean shift:
To address the problem of the fixed number of the cluster k, the mean shift algorithm (Comaniciu et al., 2002) is applied by Crandall et al. (2009) to find locations on metropolitan and landmark scales. Instead of fixing the number of clusters, mean shift requires a bandwidth parameter to specify the density radius, which can be viewed as the influential area of a location. Crandall et al. (2009) set the bandwidth subjectively to 100 km for the metropolitan-scale and 100 m for the landmark scale. Same as k-means, the mean shift is also a mean-based clustering method that will assign a mean location for each cluster. To discover sightseeing spots, one straightforward solution is to set a small bandwidth value globally for all locations. However, it is very hard to find a suitable value for different cities precisely, and the unified bandwidth value for all locations has also suffered the problem of different scales of sightseeing spots.

DBSCAN:
Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is a common densitybased clustering algorithm for spatial data (Ester et al., 1996). The algorithm adopts a range search strategy that requires two parameters. One is the search range radius and another is to determine the minimum number of locations needed to form a cluster. The benefits of DBSCAN compared to the above clustering algorithms are the high robustness against outliers and there is no constraint on the shape of clusters. P-DBSCAN (Kisilevich et al., 2010) adds an additional local density ratio parameter to restrict reachability between locations. POI or AOI identification results by performing DBSCAN and P-DBSCAN are reported in Yang et al. (2011) andLaptev et al. (2014), which reveals that the large areas produced by DBSCAN are split into parts of almost constant densities. However, since the algorithm cannot provide the coordinates of the output clusters, further computation is needed when using it for the sightseeing spot discovery task. Also, since most of the sightseeing spots we want to find are located in areas with high photo density, it is hard to find suitable parameters for the range search.
Other clustering algorithms are also applied to the POI/AOI discovery task. For example, Yang et al. (2011) proposed a parameter-free method based on spectral clustering (Bühler et al., 2009) to identify POIs, and OPTISC (Ankerst et al., 2000), which is another density-based clustering algorithm, was applied to discover AOIs by Liu et al. (2012). Similar to DBSCAN, the results of these two studies cannot provide a precise representative coordinate for each cluster and neither considers the problem of different granularities of POIs. Both regions of interest (ROI) and POI are considered in Kuo et al. (2018). In this work, however, the POI is only the representative point of the discovered ROI, which is different from the granularities between the AOI and the sightseeing spots discussed in this article. In this work, we apply and analyse LBSN data to improve the location recommendation services which broadly fall within the scope of social informatics ( Fusco et al., 2010;Masami, 2014). This article is an extended and reworked version of the conference paper (Sun et al., 2020).
The rest of this article is organized as follow: Section 2 introduces our proposed framework and Section 3 provides the experiments results and discussion. Finally, we briefly conclude our work in Section 4.

Methods
According to the discussion in the literature review, different clustering methods have advantages and disadvantages when applied to the POI discovery task. Therefore, we combine two clustering algorithms to deal with POIs of different granularities, and the parameters are determined automatically by a city adaptive tuning process.

AOI discovery
We first detect AOIs according to the photo density. We aim to eliminate low photo density areas in the city that may not be worth visiting. This can be done by applying the DBSCAN algorithm, which can detect clusters of various sizes and shapes, and performs better than the mean-based clustering algorithms (Lee et al., 2013). The reason why we adopt DBSCAN rather than its modification P-DBSCAN is that we want to find high photo density areas no matter how large they are. For instance, scenic paths or shopping streets might be separated into pieces by P-DBSCAN clustering because they probably do not reach a constant density.

DBSCAN takes two parameters, and
, and it is difficult to determine appropriate values since different cities have different parameter settings. In order to determine suitable parameters for various datasets automatically, we propose a self-tuning process for AOI discovery. To be adaptive to different cities, we can use the location and size information of several typical AOIs in the city. We use them as the density criterion by comparing the coverage degree of detected clusters with typical AOI area information.
We use the minimum bounding rectangle (MBR) to represents the area of the typical AOI and the Dice coefficient to estimate the cover area. The higher the similarity between the cluster detected by DBSCAN clustering and the predetermined AOI rectangle, the higher the accuracy of clustering is considered. In this way, the parameter is tuned by the overlapping area of both areas, and high photo density AOIs are detected.
Suppose MBRs = { 1 , 2 , … , } are determined with typical AOIs in advance. The area of the rectangle is denoted by . For each of the clusters = { 1 , 2 , … , } obtained by DBSCAN clustering, we can find the smallest rectangle = { 1 , 2 , . . . , } covering c. The area of the rectangle is denoted by . Let , be the overlap rectangle between and , and we define its area as , . Therefore, the Dice coefficients of the two areas are defined as below: ( , ) = 2 , + Since there may be multiple rectangles covering the same AOI, we only use the highest one to represent the Dice coefficient of the typical AOI as follows: Finally, the accuracy score for the parameters and are defined as below: We choose the parameters with the highest accuracy scores to discover AOIs. The discovered AOIs can be represented by MBRs or polygons by using the GPS coordinates in each cluster. An outline of the tuning process is also shown in Figure 2.

Sightseeing spot discovery
Mean shift clustering is adopted to discover sightseeing spots from AOIs, which are detected by DBSCAN clustering in the first step. There are two benefits of using mean shift clustering. One is that photos are usually taken around the spot, so we can use the Gaussian kernel for mean shift clustering. The other is that a mean will be assigned to each cluster, which can be treated as a detected sightseeing spot. We find sightseeing spots in each AOI instead of applying mean shift to all data points. The benefit of this is that low photo density areas are avoided. Also, the detected sightseeing spots are inside the AOIs, which are easier for constructing the inside routes.
Mean shift clustering is an iterative process in which every data point moves towards a clustering mean at each iteration. Accordingly, for data points in each AOI, we mark the clustering mean as the sightseeing spot that is estimated by: where ( ) is the neighbourhood of data point and ( − ) is the kernel function. We use the Gaussian kernel (i.e., ( − ) = 1 √2 ℎ 2 − ‖ − ‖ 2 2ℎ 2 ) as the kernel function that requires the neighbourhood of to satisfy ∥ − ∥ ≤ ℎ, where ℎ is the bandwidth parameter. The bandwidth parameter can be viewed as the influential area of a sightseeing spot (i.e., clustering centre). However, it is not appropriate to take the same bandwidth for all AOIs since the density of photos and the AOI area vary widely. We propose an area-adaptive parameter setting method to automatically determine the bandwidth for each AOI.
For simplicity, we consider two main factors that affect sightseeing spot discovery in AOIs: • AOI area: The bandwidth could be proportional to the AOI area. For instance, the bandwidth of a huge park should be larger than that of a small museum. We use the smallest cover rectangle area defined in 2.1 to represent the area of the AOI .
• Photo quantity: The bandwidth could be inversely proportional to photo quantity in the AOI. For instance, a high photo density area contains many more sightseeing spots and setting a smaller bandwidth could find more spots. We use to represent the number of photos in the AOI .
Therefore, we define inverse photo density to represent the relation of the two factors above for each AOI as below: = Finally, we define the bandwidth parameter ℎ for each AOI as below: where ℎ is used as the hyperparameter to make sure the range of the bandwidth is less than ℎ ; ′ = max ( ) , which normalizes the inverse photo density to the range of (0, 1].
We use the automatically determined bandwidth ℎ to apply to mean shift clustering for each AOI.
Consequently, we mark the clustering centre as the sightseeing spot. Then, the relevant information of the detected sightseeing spots for tour recommendation applications can be obtained easily. For example, the spot visit duration can be obtained by calculating the average value of the time difference between the last photo and the earliest photo taken by the tourists on the same day.

Experiments
We apply our proposed framework to two geo-tagged photo datasets. Both quantitative evaluation and case study analysis are presented to demonstrate our findings. Also, we show an example of applying the POIs with different granularities discovered by our framework for tour recommendation.
We collect 231,245 geo-tagged photos taken in Kyoto, Japan, and 271,081 photos taken in Paris, France from Flickr API 1 to discover sightseeing spots. Both cities are famous tourist destinations but different in culture. We extract 20 MBRs from OpenStreetMap 2 , 10 MBRs are used for initial parameter selection and the remaining 10 are used as the validation set, in which we mark sightseeing spots inside the MBRs manually as the ground truth (104 spots in Kyoto and 72 spots in Paris). One location is marked as a sightseeing spot based on the symbol on the map (e.g., viewpoint, shrine, etc.).
A grid search strategy is adopted to find the most suitable parameters with the highest dice coefficient for the AOI discovery phase. We set ℎ = 50 metres as the range of the bandwidth hyperparameter for sightseeing spot discovery since we want to find fine-grained POIs. We test different parameter settings for various baseline methods on both datasets.

Parameter tuning
Since we use 10 MBRs of typical AOIs for city adaptive parameter selection, we examine the impact of different sets of MBRs. Two sets of MBRs are tested, one is composed of the most popular sightseeing attractions in Kyoto, the other is composed of common attractions that are less well known but highly rated by locals. The tuning processes are shown in Figure 3. We can find that MBRs of popular attractions tend to choose a more aggressive search range by a large value of because of the high photo quantity density in those MBRs. The results of AOIs discovered by different sets of MBRs are shown in Figure 4. We find that fewer clusters are discovered by using MBRs of popular AOIs since the required parameters are relatively higher. In contrast, more potential AOIs can be found by using a set of MBRs of common attractions. This reveals that it is very hard to set a perfect parameter to find all AOIs. In our framework, we can use different sets of MBRs to discover AOIs more specifically.

Performance evaluation on sightseeing spot discovery
We compare our framework against the various baselines mentioned in Section 1.1 on two datasets using the following metrics: • Number of Spots: The total number of discovered sightseeing spots.
• Precision: The proportion of discovered sightseeing spots inside the validation MBRs that also exist in the ground truth. Considering that the photos may be taken around the spots, we regard the points closest to the ground truth and not more than 10 metres away as the same. • Recall: The proportion of sightseeing spots in the ground truth that also exists in the discovered spots inside the validation MBRs. Also, we regard those closest points that are less than 10 metres away from the ground truth as the same. • F-score: The harmonic mean of the precision and recall mentioned above. The recall of the mean shift with the fixed bandwidth parameter decreases as the value of bandwidth increases, while the precision increases with increasing bandwidth, as shown in Figure 5(a) and Figure  6(a). The reason is that more spots can be found when setting a smaller bandwidth value, so the recall is higher at first and drops as the bandwidth value increases. We vary the parameter k in k-means clustering and the result is reported in Figure 5(b) and Figure 6(b). Obviously, it is hard to decide an appropriate value for the parameter k, especially for this sightseeing spot discovery task, as the number of spots will usually be large. When k is large enough, the recall is getting better since more spots are marked as sightseeing spots. However, this also leads to more non-sightseeing spots being included, which would reduce the total quality of the discovered spots.   For DBSCAN, we set = 2 for the Kyoto dataset and = 5 for Paris dataset due to the difference in photo quantity, only changing the search range parameter epsilon. The spot is computed by the mean value of the cluster since it does not explicitly return cluster centres. Contrary to mean shift, when the search range ε is small, DBSCAN only finds the densest areas, which leads to a high recall at first, as shown in Figure 5(c) and Figure 6(c). When the search range gets bigger, the recall drops quickly.
According to the results in Table 1, the mean shift finds the greatest number of spots since it is applied to all data points with a small influential area. It is hard to choose a proper value of k for the k-means clustering, and the results vary on different datasets. As shown in Figure 5 and Figure 6, the results of the baseline methods are greatly affected by the value of parameters. Appropriate parameters also vary depending on the datasets. Our methods rarely depend on parameters and get stable results.
Note that we set a fixed bandwidth of h = 20 on the discovered AOIs, and get a better result that compares to mean shift clustering applied to all data points. It indicates the effectiveness of our idea to apply DBSCAN to remove noises and outliers. As a result, the number of spots produced by our method is relatively smaller on both datasets. Overall, our proposed framework achieves better results in terms of F-score on the average of two datasets, which verifies the effectiveness of the city adaptive parameter tuning strategy in our framework.

Case study
The Nijo Castle is a famous sightseeing attraction in Kyoto. We show the sightseeing spots around the Nijo Castle discovered by different methods in Figure 7, where the red points in each figure represent the detected sightseeing spots.
First of all, we can see both k-means, mean shift and our method mark the spots inside the Nijo Castle very well, and most of the discovered spots are at the centre of a high photo density area according to the heat map on the right side of Figure 1. Note that the value of the bandwidth parameter in our method is determined for this area automatically. However, many spots outside the Nijo Castle, which do not seem to be sightseeing attractions, are also marked as sightseeing spots by k-means and mean shift. It implies that discovering sightseeing spots on all photo data directly with mean-based clustering algorithms will produce a lot of noise, which reduces the quality of the discovered spots. On the contrary, most of the spots produced by DBSCAN are located inside the Nijo Castle because it removes low-density data points that do not meet the density requirement. Since no explicit cluster centre is returned by DBSCAN, the location of the discovered sightseeing spots does not seem accurate enough. From the results, we can easily find out the advantages of combining the density-based and mean-based clustering techniques on the sightseeing spot discovery task.
We show more examples of discovered sightseeing spots in the Paris datasets in Figure 8. We can see that most of the marked sightseeing spots are located at the centre of the structure. Some places such as the gate and atrium where tourists would like to take photos are also marked out. With this location information, the tour planning algorithm can recommend a detailed travel route for users.

Tour recommendation
With discovered POIs of different granularities, we apply them to a tour recommendation application introduced by Wang et al. (2018). It combines outside and inside route planning to recommend a detailed travel route for tourists. We use the discovered AOIs for outside route planning and sightseeing spots for inside route planning. For simplicity, an average speed of 5 km/h is set as the transport speed between locations. The profit of the location is computed by the photo quantity in the cluster we found and the visit duration is computed with the time interval of two photos produced by the same user on one spot.
For instance, a recommended tour is shown in Figure 9. By using the discovered POIs with different granularities, the recommended tour combines areas and points that provides much more detailed information for tourists. Therefore, we think this information and the sightseeing spots we found would be helpful for tourists travelling to an unfamiliar destination.

Conclusion and Future Work
In this article, we summarize various clustering techniques for the POI discovery task and propose a city adaptive clustering framework to discover POIs with different granularities for tour recommendation services. The proposed method combines the advantages of two clustering algorithms and the parameters are determined automatically according to different cities. We compare our proposed framework against various baselines with careful parameter settings. The experimental results on two famous travel destinations reveal the effectiveness of our proposed framework. In addition, we apply the discovered POIs of different granularities to tour recommendation and the recommended tour has been improved by providing both outside and detailed inside routes.
With the framework proposed in this article, we intend to collect images of different popular travel destination cities and publish a public fine-grained POI dataset for various POI recommendation tasks. Also, we would like to propose a fine-grained tour recommendation method to improve recommendation performance on the released public dataset.

Additional Information and Declarations
Funding: This work is partly supported by MIC SCOPE (172307001, 201607008).