org.apache.marmotta.ldclient.provider.html

Class AbstractHTMLDataProvider

java.lang.Object

org.apache.marmotta.ldclient.services.provider.AbstractHttpProvider

org.apache.marmotta.ldclient.provider.html.AbstractHTMLDataProvider

All Implemented Interfaces:

DataProvider

public abstract class AbstractHTMLDataProvider
extends AbstractHttpProvider
implements DataProvider

Generic implementation of an HTML data provider capable of mapping XPath expressions from HTML documents to RDF metadata properties. The HTML data provider will first clean up the potentially messy HTML using HTMLCleaner and then evaluate XPath expressions on it in the same way as the AbstractXMLDataProvider

Author: Sebastian Schaffert

Field Summary

Fields inherited from class org.apache.marmotta.ldclient.services.provider.AbstractHttpProvider

RETRY_AFTER

Constructor Summary

Constructors

Constructor and Description
AbstractHTMLDataProvider()

Method Summary

Methods

Modifier and Type	Method and Description
protected List<String>	findAdditionalRequestUrls(String resource, org.jsoup.nodes.Document document, String requestUrl) Try to find further URLs in the document that need to be requested to complete the resource data.
protected abstract Map<String,JSoupMapper>	getMappings(String resource, String requestUrl)
protected abstract List<String>	getTypes(org.openrdf.model.URI resource) Return a list of URIs that should be added as types for each processed resource.
List<String>	parseResponse(String resource, String requestUrl, org.openrdf.model.Model triples, InputStream in, String contentType) Parse the HTTP response entity returned by the web service call and return its contents as a Sesame RDF repository.

Methods inherited from class org.apache.marmotta.ldclient.services.provider.AbstractHttpProvider

buildRequestUrl, isValidContentType, retrieveResource

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.marmotta.ldclient.api.provider.DataProvider

getName, listMimeTypes, retrieveResource

Constructor Detail

AbstractHTMLDataProvider

public AbstractHTMLDataProvider()

Method Detail

getTypes

protected abstract List<String> getTypes(org.openrdf.model.URI resource)

Return a list of URIs that should be added as types for each processed resource.

Parameters:

resource -

Returns:

findAdditionalRequestUrls

protected List<String> findAdditionalRequestUrls(String resource,
                                     org.jsoup.nodes.Document document,
                                     String requestUrl)

Try to find further URLs in the document that need to be requested to complete the resource data. Used e.g. to parse the result of paging in HTML pages. The default implementation returns an empty list.

Parameters:

resource -

document -

requestUrl -

Returns:

parseResponse

public List<String> parseResponse(String resource,
                         String requestUrl,
                         org.openrdf.model.Model triples,
                         InputStream in,
                         String contentType)
                           throws DataRetrievalException

Parse the HTTP response entity returned by the web service call and return its contents as a Sesame RDF repository. The content type returned by the web service is passed as argument to help the implementation decide how to parse the data.

Specified by:

parseResponse in class AbstractHttpProvider

Parameters:

resource - the subject of the data retrieval

triples -

in - input stream as returned by the remote webservice

contentType - content type as returned in the HTTP headers of the remote webservice @return an RDF repository containing an RDF representation of the dataset located at the remote resource.

Returns:

a possibly empty list of URLs of additional resources to retrieve to complete the content

Throws:

IOException - in case an error occurs while reading the input stream

DataRetrievalException

getMappings

protected abstract Map<String,JSoupMapper> getMappings(String resource,
                                  String requestUrl)