[SCM] WebKit Debian packaging branch, debian/experimental, updated. upstream/1.3.3-9427-gc2be6fc
joepeck at webkit.org
joepeck at webkit.org
Wed Dec 22 11:37:09 UTC 2010
The following commit has been merged in the debian/experimental branch:
commit 598a6948333593762057e6b90444838e640a2474
Author: joepeck at webkit.org <joepeck at webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Date: Sat Jul 31 02:22:16 2010 +0000
2010-07-30 Joseph Pecoraro <joepeck at webkit.org>
Reviewed by David Kilzer.
Limit ApplicationCache Total and Per-Origin Storage Capacity (Quotas)
https://bugs.webkit.org/show_bug.cgi?id=40627
Part 2 - Update Schema and enforce Per-Origin Quotas
Added an "Origins" table to the application cache databases.
This, like the Database's Origins table, is a list of origin
and quota pairs. Origins records are added as soon as they are
needed, and deleted only when the ApplicationCacheStorage is
emptied. This means Origins records persist even after all
caches for that origin may be deleted. The "CacheGroups" table
now has a foreign key column "origin" which relates to the
"Origins" table.
To enforce the quotas, remaining quota space is checked at
the start of update as an estimate and at the end before
inserting. Currently, reaching the quota limit will simply
cause an update error. A later part will provide a
notification to the client to allow an action, and refactor
the final quota limit check into a transaction.
Respect the quota during the update process. And cause
the update process to fail when the quota is reached.
* loader/appcache/ApplicationCacheGroup.cpp: added loading counter, counts bytes as they load
(WebCore::ApplicationCacheGroup::ApplicationCacheGroup):
(WebCore::ApplicationCacheGroup::didReceiveData):
(WebCore::ApplicationCacheGroup::didFinishLoading):
(WebCore::ApplicationCacheGroup::checkIfLoadIsComplete):
* loader/appcache/ApplicationCacheGroup.h: added security origin, based on the manifest URL
(WebCore::ApplicationCacheGroup::origin): accessor
Updates the schema of the database tables as described
above. Handle other SQL operations such as checking the
remaining space and inserting and deleting Origins records.
* loader/appcache/ApplicationCacheStorage.cpp:
(WebCore::ApplicationCacheStorage::quotaForOrigin): query for the quota of an origin, may return the default origin quota if it didn't exist.
(WebCore::ApplicationCacheStorage::remainingSizeForOriginExcludingCache): calculate the remaining size in a quota for an origin, possibly excluding a cache.
(WebCore::ApplicationCacheStorage::storeUpdatedQuotaForOrigin): persistent update.
(WebCore::ApplicationCacheStorage::openDatabase): updated schema for CachesGroups, added new table Origins.
(WebCore::ApplicationCacheStorage::empty): wipe Origins table as well.
(WebCore::ApplicationCacheStorage::unknownQuota): constant to mean unknown quota
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@64398 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/WebCore/ChangeLog b/WebCore/ChangeLog
index 5c0a0b7..dcf78b0 100644
--- a/WebCore/ChangeLog
+++ b/WebCore/ChangeLog
@@ -5,6 +5,54 @@
Limit ApplicationCache Total and Per-Origin Storage Capacity (Quotas)
https://bugs.webkit.org/show_bug.cgi?id=40627
+ Part 2 - Update Schema and enforce Per-Origin Quotas
+
+ Added an "Origins" table to the application cache databases.
+ This, like the Database's Origins table, is a list of origin
+ and quota pairs. Origins records are added as soon as they are
+ needed, and deleted only when the ApplicationCacheStorage is
+ emptied. This means Origins records persist even after all
+ caches for that origin may be deleted. The "CacheGroups" table
+ now has a foreign key column "origin" which relates to the
+ "Origins" table.
+
+ To enforce the quotas, remaining quota space is checked at
+ the start of update as an estimate and at the end before
+ inserting. Currently, reaching the quota limit will simply
+ cause an update error. A later part will provide a
+ notification to the client to allow an action, and refactor
+ the final quota limit check into a transaction.
+
+ Respect the quota during the update process. And cause
+ the update process to fail when the quota is reached.
+
+ * loader/appcache/ApplicationCacheGroup.cpp: added loading counter, counts bytes as they load
+ (WebCore::ApplicationCacheGroup::ApplicationCacheGroup):
+ (WebCore::ApplicationCacheGroup::didReceiveData):
+ (WebCore::ApplicationCacheGroup::didFinishLoading):
+ (WebCore::ApplicationCacheGroup::checkIfLoadIsComplete):
+ * loader/appcache/ApplicationCacheGroup.h: added security origin, based on the manifest URL
+ (WebCore::ApplicationCacheGroup::origin): accessor
+
+ Updates the schema of the database tables as described
+ above. Handle other SQL operations such as checking the
+ remaining space and inserting and deleting Origins records.
+
+ * loader/appcache/ApplicationCacheStorage.cpp:
+ (WebCore::ApplicationCacheStorage::quotaForOrigin): query for the quota of an origin, may return the default origin quota if it didn't exist.
+ (WebCore::ApplicationCacheStorage::remainingSizeForOriginExcludingCache): calculate the remaining size in a quota for an origin, possibly excluding a cache.
+ (WebCore::ApplicationCacheStorage::storeUpdatedQuotaForOrigin): persistent update.
+ (WebCore::ApplicationCacheStorage::openDatabase): updated schema for CachesGroups, added new table Origins.
+ (WebCore::ApplicationCacheStorage::empty): wipe Origins table as well.
+ (WebCore::ApplicationCacheStorage::unknownQuota): constant to mean unknown quota
+
+2010-07-30 Joseph Pecoraro <joepeck at webkit.org>
+
+ Reviewed by David Kilzer.
+
+ Limit ApplicationCache Total and Per-Origin Storage Capacity (Quotas)
+ https://bugs.webkit.org/show_bug.cgi?id=40627
+
Part 1 - Add Total and Per-Origin Quota Preferences.
Allow the application cache total size to be a preference, and
diff --git a/WebCore/loader/appcache/ApplicationCacheGroup.cpp b/WebCore/loader/appcache/ApplicationCacheGroup.cpp
index 9b05e0d..a8687c6 100644
--- a/WebCore/loader/appcache/ApplicationCacheGroup.cpp
+++ b/WebCore/loader/appcache/ApplicationCacheGroup.cpp
@@ -42,6 +42,7 @@
#include "MainResourceLoader.h"
#include "ManifestParser.h"
#include "Page.h"
+#include "SecurityOrigin.h"
#include "Settings.h"
#include <wtf/HashMap.h>
@@ -57,6 +58,7 @@ namespace WebCore {
ApplicationCacheGroup::ApplicationCacheGroup(const KURL& manifestURL, bool isCopy)
: m_manifestURL(manifestURL)
+ , m_origin(SecurityOrigin::create(manifestURL))
, m_updateStatus(Idle)
, m_downloadingPendingMasterResourceLoadersCount(0)
, m_progressTotal(0)
@@ -67,6 +69,8 @@ ApplicationCacheGroup::ApplicationCacheGroup(const KURL& manifestURL, bool isCop
, m_completionType(None)
, m_isCopy(isCopy)
, m_calledReachedMaxAppCacheSize(false)
+ , m_loadedSize(0)
+ , m_availableSpaceInQuota(ApplicationCacheStorage::unknownQuota())
{
}
@@ -592,6 +596,8 @@ void ApplicationCacheGroup::didReceiveData(ResourceHandle* handle, const char* d
ASSERT(m_currentResource);
m_currentResource->data()->append(data, length);
+
+ m_loadedSize += length;
}
void ApplicationCacheGroup::didFinishLoading(ResourceHandle* handle)
@@ -605,7 +611,23 @@ void ApplicationCacheGroup::didFinishLoading(ResourceHandle* handle)
didFinishLoadingManifest();
return;
}
-
+
+ // After finishing the loading of any resource, we check if it will
+ // fit in our last known quota limit.
+ if (m_availableSpaceInQuota == ApplicationCacheStorage::unknownQuota()) {
+ // Failed to determine what is left in the quota. Fallback to allowing anything.
+ if (!cacheStorage().remainingSizeForOriginExcludingCache(m_origin.get(), m_newestCache.get(), m_availableSpaceInQuota))
+ m_availableSpaceInQuota = ApplicationCacheStorage::noQuota();
+ }
+
+ // Check each resource, as it loads, to see if it would fit in our
+ // idea of the available quota space.
+ if (m_availableSpaceInQuota < m_loadedSize) {
+ m_currentResource = 0;
+ cacheUpdateFailed();
+ return;
+ }
+
ASSERT(m_currentHandle == handle);
ASSERT(m_pendingEntries.contains(handle->firstRequest().url()));
@@ -860,6 +882,15 @@ void ApplicationCacheGroup::checkIfLoadIsComplete()
RefPtr<ApplicationCache> oldNewestCache = (m_newestCache == m_cacheBeingUpdated) ? 0 : m_newestCache;
+ // Check one more time, before committing to the new cache, if the cache will fit in the quota.
+ int64_t remainingSpaceInOrigin;
+ if (cacheStorage().remainingSizeForOriginExcludingCache(m_origin.get(), oldNewestCache.get(), remainingSpaceInOrigin)) {
+ if (m_cacheBeingUpdated->estimatedSizeInStorage() > remainingSpaceInOrigin) {
+ cacheUpdateFailed();
+ break;
+ }
+ }
+
setNewestCache(m_cacheBeingUpdated.release());
if (cacheStorage().storeNewestCache(this)) {
// New cache stored, now remove the old cache.
@@ -922,6 +953,8 @@ void ApplicationCacheGroup::checkIfLoadIsComplete()
m_completionType = None;
setUpdateStatus(Idle);
m_frame = 0;
+ m_loadedSize = 0;
+ m_availableSpaceInQuota = ApplicationCacheStorage::unknownQuota();
m_calledReachedMaxAppCacheSize = false;
}
diff --git a/WebCore/loader/appcache/ApplicationCacheGroup.h b/WebCore/loader/appcache/ApplicationCacheGroup.h
index 9b8a6ab..9b68ccc 100644
--- a/WebCore/loader/appcache/ApplicationCacheGroup.h
+++ b/WebCore/loader/appcache/ApplicationCacheGroup.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2008, 2009 Apple Inc. All Rights Reserved.
+ * Copyright (C) 2008, 2009, 2010 Apple Inc. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -28,10 +28,6 @@
#if ENABLE(OFFLINE_WEB_APPLICATIONS)
-#include <wtf/Noncopyable.h>
-#include <wtf/HashMap.h>
-#include <wtf/HashSet.h>
-
#include "DOMApplicationCache.h"
#include "KURL.h"
#include "PlatformString.h"
@@ -39,6 +35,10 @@
#include "ResourceHandleClient.h"
#include "SharedBuffer.h"
+#include <wtf/Noncopyable.h>
+#include <wtf/HashMap.h>
+#include <wtf/HashSet.h>
+
namespace WebCore {
class ApplicationCache;
@@ -46,6 +46,7 @@ class ApplicationCacheResource;
class Document;
class DocumentLoader;
class Frame;
+class SecurityOrigin;
enum ApplicationCacheUpdateOption {
ApplicationCacheUpdateWithBrowsingContext,
@@ -66,6 +67,7 @@ public:
static void selectCacheWithoutManifestURL(Frame*);
const KURL& manifestURL() const { return m_manifestURL; }
+ const SecurityOrigin* origin() const { return m_origin.get(); }
UpdateStatus updateStatus() const { return m_updateStatus; }
void setUpdateStatus(UpdateStatus status);
@@ -131,6 +133,7 @@ private:
void stopLoading();
KURL m_manifestURL;
+ RefPtr<SecurityOrigin> m_origin;
UpdateStatus m_updateStatus;
// This is the newest complete cache in the group.
@@ -194,6 +197,9 @@ private:
RefPtr<ApplicationCacheResource> m_manifestResource;
RefPtr<ResourceHandle> m_manifestHandle;
+ int64_t m_loadedSize;
+ int64_t m_availableSpaceInQuota;
+
friend class ChromeClientCallbackTimer;
};
diff --git a/WebCore/loader/appcache/ApplicationCacheStorage.cpp b/WebCore/loader/appcache/ApplicationCacheStorage.cpp
index 5557d40..8842142 100644
--- a/WebCore/loader/appcache/ApplicationCacheStorage.cpp
+++ b/WebCore/loader/appcache/ApplicationCacheStorage.cpp
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2008, 2009 Apple Inc. All Rights Reserved.
+ * Copyright (C) 2008, 2009, 2010 Apple Inc. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -29,13 +29,14 @@
#if ENABLE(OFFLINE_WEB_APPLICATIONS)
#include "ApplicationCache.h"
-#include "ApplicationCacheHost.h"
#include "ApplicationCacheGroup.h"
+#include "ApplicationCacheHost.h"
#include "ApplicationCacheResource.h"
#include "FileSystem.h"
#include "KURL.h"
#include "SQLiteStatement.h"
#include "SQLiteTransaction.h"
+#include "SecurityOrigin.h"
#include <wtf/text/CString.h>
#include <wtf/StdLibExtras.h>
#include <wtf/StringExtras.h>
@@ -419,6 +420,94 @@ void ApplicationCacheStorage::setDefaultOriginQuota(int64_t quota)
m_defaultOriginQuota = quota;
}
+bool ApplicationCacheStorage::quotaForOrigin(const SecurityOrigin* origin, int64_t& quota)
+{
+ // If an Origin record doesn't exist, then the COUNT will be 0 and quota will be 0.
+ // Using the count to determine if a record existed or not is a safe way to determine
+ // if a quota of 0 is real, from the record, or from null.
+ SQLiteStatement statement(m_database, "SELECT COUNT(quota), quota FROM Origins WHERE origin=?");
+ if (statement.prepare() != SQLResultOk)
+ return false;
+
+ statement.bindText(1, origin->databaseIdentifier());
+ int result = statement.step();
+
+ // Return the quota, or if it was null the default.
+ if (result == SQLResultRow) {
+ bool wasNoRecord = statement.getColumnInt64(0) == 0;
+ quota = wasNoRecord ? m_defaultOriginQuota : statement.getColumnInt64(1);
+ return true;
+ }
+
+ LOG_ERROR("Could not get the quota of an origin, error \"%s\"", m_database.lastErrorMsg());
+ return false;
+}
+
+bool ApplicationCacheStorage::remainingSizeForOriginExcludingCache(const SecurityOrigin* origin, ApplicationCache* cache, int64_t& remainingSize)
+{
+ openDatabase(false);
+ if (!m_database.isOpen())
+ return false;
+
+ // Remaining size = total origin quota - size of all caches with origin excluding the provided cache.
+ // Keep track of the number of caches so we can tell if the result was a calculation or not.
+ const char* query;
+ int64_t excludingCacheIdentifier = cache ? cache->storageID() : 0;
+ if (excludingCacheIdentifier != 0) {
+ query = "SELECT COUNT(Caches.size), Origins.quota - SUM(Caches.size)"
+ " FROM CacheGroups"
+ " INNER JOIN Origins ON CacheGroups.origin = Origins.origin"
+ " INNER JOIN Caches ON CacheGroups.id = Caches.cacheGroup"
+ " WHERE Origins.origin=?"
+ " AND Caches.id!=?";
+ } else {
+ query = "SELECT COUNT(Caches.size), Origins.quota - SUM(Caches.size)"
+ " FROM CacheGroups"
+ " INNER JOIN Origins ON CacheGroups.origin = Origins.origin"
+ " INNER JOIN Caches ON CacheGroups.id = Caches.cacheGroup"
+ " WHERE Origins.origin=?";
+ }
+
+ SQLiteStatement statement(m_database, query);
+ if (statement.prepare() != SQLResultOk)
+ return false;
+
+ statement.bindText(1, origin->databaseIdentifier());
+ if (excludingCacheIdentifier != 0)
+ statement.bindInt64(2, excludingCacheIdentifier);
+ int result = statement.step();
+
+ // If the count was 0 that then we have to query the origin table directly
+ // for its quota. Otherwise we can use the calculated value.
+ if (result == SQLResultRow) {
+ int64_t numberOfCaches = statement.getColumnInt64(0);
+ if (numberOfCaches == 0)
+ quotaForOrigin(origin, remainingSize);
+ else
+ remainingSize = statement.getColumnInt64(1);
+ return true;
+ }
+
+ LOG_ERROR("Could not get the remaining size of an origin's quota, error \"%s\"", m_database.lastErrorMsg());
+ return false;
+}
+
+bool ApplicationCacheStorage::storeUpdatedQuotaForOrigin(const SecurityOrigin* origin, int64_t quota)
+{
+ openDatabase(false);
+ if (!m_database.isOpen())
+ return false;
+
+ SQLiteStatement updateStatement(m_database, "UPDATE Origins SET quota=? WHERE origin=?");
+ if (updateStatement.prepare() != SQLResultOk)
+ return false;
+
+ updateStatement.bindInt64(1, quota);
+ updateStatement.bindText(2, origin->databaseIdentifier());
+
+ return executeStatement(updateStatement);
+}
+
bool ApplicationCacheStorage::executeSQLCommand(const String& sql)
{
ASSERT(m_database.isOpen());
@@ -431,7 +520,10 @@ bool ApplicationCacheStorage::executeSQLCommand(const String& sql)
return result;
}
-static const int schemaVersion = 5;
+// Update the schemaVersion when the schema of any the Application Cache
+// SQLite tables changes. This allows the database to be rebuilt when
+// a new, incompatible change has been introduced to the database schema.
+static const int schemaVersion = 6;
void ApplicationCacheStorage::verifySchemaVersion()
{
@@ -480,7 +572,7 @@ void ApplicationCacheStorage::openDatabase(bool createIfDoesNotExist)
// Create tables
executeSQLCommand("CREATE TABLE IF NOT EXISTS CacheGroups (id INTEGER PRIMARY KEY AUTOINCREMENT, "
- "manifestHostHash INTEGER NOT NULL ON CONFLICT FAIL, manifestURL TEXT UNIQUE ON CONFLICT FAIL, newestCache INTEGER)");
+ "manifestHostHash INTEGER NOT NULL ON CONFLICT FAIL, manifestURL TEXT UNIQUE ON CONFLICT FAIL, newestCache INTEGER, origin TEXT)");
executeSQLCommand("CREATE TABLE IF NOT EXISTS Caches (id INTEGER PRIMARY KEY AUTOINCREMENT, cacheGroup INTEGER, size INTEGER)");
executeSQLCommand("CREATE TABLE IF NOT EXISTS CacheWhitelistURLs (url TEXT NOT NULL ON CONFLICT FAIL, cache INTEGER NOT NULL ON CONFLICT FAIL)");
executeSQLCommand("CREATE TABLE IF NOT EXISTS CacheAllowsAllNetworkRequests (wildcard INTEGER NOT NULL ON CONFLICT FAIL, cache INTEGER NOT NULL ON CONFLICT FAIL)");
@@ -490,6 +582,7 @@ void ApplicationCacheStorage::openDatabase(bool createIfDoesNotExist)
executeSQLCommand("CREATE TABLE IF NOT EXISTS CacheResources (id INTEGER PRIMARY KEY AUTOINCREMENT, url TEXT NOT NULL ON CONFLICT FAIL, "
"statusCode INTEGER NOT NULL, responseURL TEXT NOT NULL, mimeType TEXT, textEncodingName TEXT, headers TEXT, data INTEGER NOT NULL ON CONFLICT FAIL)");
executeSQLCommand("CREATE TABLE IF NOT EXISTS CacheResourceData (id INTEGER PRIMARY KEY AUTOINCREMENT, data BLOB)");
+ executeSQLCommand("CREATE TABLE IF NOT EXISTS Origins (origin TEXT UNIQUE ON CONFLICT IGNORE, quota INTEGER NOT NULL ON CONFLICT FAIL)");
// When a cache is deleted, all its entries and its whitelist should be deleted.
executeSQLCommand("CREATE TRIGGER IF NOT EXISTS CacheDeleted AFTER DELETE ON Caches"
@@ -528,16 +621,31 @@ bool ApplicationCacheStorage::store(ApplicationCacheGroup* group, GroupStorageID
ASSERT(group->storageID() == 0);
ASSERT(journal);
- SQLiteStatement statement(m_database, "INSERT INTO CacheGroups (manifestHostHash, manifestURL) VALUES (?, ?)");
+ SQLiteStatement statement(m_database, "INSERT INTO CacheGroups (manifestHostHash, manifestURL, origin) VALUES (?, ?, ?)");
if (statement.prepare() != SQLResultOk)
return false;
+ String originIdentifier = group->origin()->databaseIdentifier();
+
statement.bindInt64(1, urlHostHash(group->manifestURL()));
statement.bindText(2, group->manifestURL());
+ statement.bindText(3, originIdentifier);
if (!executeStatement(statement))
return false;
+ // Create Origin if needed.
+ {
+ SQLiteStatement insertOriginStatement(m_database, "INSERT INTO Origins (origin, quota) VALUES (?, ?)");
+ if (insertOriginStatement.prepare() != SQLResultOk)
+ return false;
+
+ insertOriginStatement.bindText(1, originIdentifier);
+ insertOriginStatement.bindInt64(2, m_defaultOriginQuota);
+ if (!executeStatement(insertOriginStatement))
+ return false;
+ }
+
group->setStorageID(static_cast<unsigned>(m_database.lastInsertRowID()));
journal->add(group, 0);
return true;
@@ -969,9 +1077,10 @@ void ApplicationCacheStorage::empty()
if (!m_database.isOpen())
return;
- // Clear cache groups, caches and cache resources.
+ // Clear cache groups, caches, cache resources, and origins.
executeSQLCommand("DELETE FROM CacheGroups");
executeSQLCommand("DELETE FROM Caches");
+ executeSQLCommand("DELETE FROM Origins");
// Clear the storage IDs for the caches in memory.
// The caches will still work, but cached resources will not be saved to disk
diff --git a/WebCore/loader/appcache/ApplicationCacheStorage.h b/WebCore/loader/appcache/ApplicationCacheStorage.h
index 342f6a9..1d43084 100644
--- a/WebCore/loader/appcache/ApplicationCacheStorage.h
+++ b/WebCore/loader/appcache/ApplicationCacheStorage.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2008 Apple Inc. All Rights Reserved.
+ * Copyright (C) 2008, 2010 Apple Inc. All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -37,12 +37,13 @@
namespace WebCore {
class ApplicationCache;
-class ApplicationCacheHost;
class ApplicationCacheGroup;
+class ApplicationCacheHost;
class ApplicationCacheResource;
class KURL;
template <class T>
class StorageIDJournal;
+class SecurityOrigin;
class ApplicationCacheStorage : public Noncopyable {
public:
@@ -56,6 +57,9 @@ public:
int64_t defaultOriginQuota() const { return m_defaultOriginQuota; }
void setDefaultOriginQuota(int64_t quota);
+ bool quotaForOrigin(const SecurityOrigin*, int64_t& quota);
+ bool remainingSizeForOriginExcludingCache(const SecurityOrigin*, ApplicationCache*, int64_t& remainingSize);
+ bool storeUpdatedQuotaForOrigin(const SecurityOrigin*, int64_t quota);
ApplicationCacheGroup* cacheGroupForURL(const KURL&); // Cache to load a main resource from.
ApplicationCacheGroup* fallbackCacheGroupForURL(const KURL&); // Cache that has a fallback entry to load a main resource from if normal loading fails.
@@ -80,6 +84,7 @@ public:
bool deleteCacheGroup(const String& manifestURL);
void vacuumDatabaseFile();
+ static int64_t unknownQuota() { return -1; }
static int64_t noQuota() { return std::numeric_limits<int64_t>::max(); }
private:
ApplicationCacheStorage();
--
WebKit Debian packaging
More information about the Pkg-webkit-commits
mailing list