diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bf87d471e34a..3d93f2d032c7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -32,6 +32,8 @@ When changes are made to authentication and project ID-related code, authenticat Known issue: If you have installed the Google Cloud SDK, be sure to log in (using `gcloud auth login`) before running tests. Though the Datastore tests use a local Datastore emulator that doesn't require authentication, they will not run if you have the Google Cloud SDK installed but aren't authenticated. +**Please, do not use your production projects for executing integration tests.** While we do our best to make our tests independent of your project's state and content, they do perform create, modify and deletes, and you do not want to have your production data accidentally modified. + Adding Features --------------- In order to add a feature to gcloud-java: diff --git a/README.md b/README.md index 207009ee6475..6061d9dd4c8f 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ Java idiomatic client for [Google Cloud Platform][cloud-platform] services. [![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) [![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java.svg) [![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) - [Homepage] (https://googlecloudplatform.github.io/gcloud-java/) - [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs) @@ -15,6 +16,7 @@ This client supports the following Google Cloud Platform services: - [Google Cloud BigQuery] (#google-cloud-bigquery-alpha) (Alpha) - [Google Cloud Datastore] (#google-cloud-datastore) +- [Google Cloud DNS] (#google-cloud-dns-alpha) (Alpha) - [Google Cloud Resource Manager] (#google-cloud-resource-manager-alpha) (Alpha) - [Google Cloud Storage] (#google-cloud-storage) @@ -28,33 +30,35 @@ If you are using Maven, add this to your pom.xml file com.google.gcloud gcloud-java - 0.1.3 + 0.1.5 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.gcloud:gcloud-java:0.1.3' +compile 'com.google.gcloud:gcloud-java:0.1.5' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.gcloud" % "gcloud-java" % "0.1.3" +libraryDependencies += "com.google.gcloud" % "gcloud-java" % "0.1.5" ``` Example Applications -------------------- - [`BigQueryExample`](./gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/BigQueryExample.java) - A simple command line interface providing some of Cloud BigQuery's functionality - - Read more about using this application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/BigQueryExample.html). + - Read more about using this application on the [`BigQueryExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/bigquery/BigQueryExample.html). - [`Bookshelf`](https://github.com/GoogleCloudPlatform/getting-started-java/tree/master/bookshelf) - An App Engine app that manages a virtual bookshelf. - This app uses `gcloud-java` to interface with Cloud Datastore and Cloud Storage. It also uses Cloud SQL, another Google Cloud Platform service. -- [`DatastoreExample`](./gcloud-java-examples/src/main/java/com/google/gcloud/examples/datastore/DatastoreExample.java) - A simple command line interface for the Cloud Datastore - - Read more about using this application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/DatastoreExample.html). +- [`DatastoreExample`](./gcloud-java-examples/src/main/java/com/google/gcloud/examples/datastore/DatastoreExample.java) - A simple command line interface for Cloud Datastore + - Read more about using this application on the [`DatastoreExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/datastore/DatastoreExample.html). +- [`DnsExample`](./gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/DnsExample.java) - A simple command line interface for Cloud DNS + - Read more about using this application on the [`DnsExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/dns/DnsExample.html). - [`ResourceManagerExample`](./gcloud-java-examples/src/main/java/com/google/gcloud/examples/resourcemanager/ResourceManagerExample.java) - A simple command line interface providing some of Cloud Resource Manager's functionality - - Read more about using this application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/ResourceManagerExample.html). + - Read more about using this application on the [`ResourceManagerExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/resourcemanager/ResourceManagerExample.html). - [`SparkDemo`](https://github.com/GoogleCloudPlatform/java-docs-samples/blob/master/managed_vms/sparkjava) - An example of using gcloud-java-datastore from within the SparkJava and App Engine Managed VM frameworks. - Read about how it works on the example's [README page](https://github.com/GoogleCloudPlatform/java-docs-samples/tree/master/managed_vms/sparkjava#how-does-it-work). - [`StorageExample`](./gcloud-java-examples/src/main/java/com/google/gcloud/examples/storage/StorageExample.java) - A simple command line interface providing some of Cloud Storage's functionality - - Read more about using this application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/StorageExample.html). + - Read more about using this application on the [`StorageExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/storage/StorageExample.html). Specifying a Project ID ----------------------- @@ -217,6 +221,71 @@ if (entity != null) { } ``` +Google Cloud DNS (Alpha) +---------------------- +- [API Documentation][dns-api] +- [Official Documentation][cloud-dns-docs] + +*Follow the [activation instructions][cloud-dns-activation] to use the Google Cloud DNS API with your project.* + +#### Preview + +Here are two code snippets showing simple usage examples from within Compute/App Engine. Note that you must [supply credentials](#authentication) and a project ID if running this snippet elsewhere. + +The first snippet shows how to create a zone resource. Complete source code can be found on +[CreateZone.java](./gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateZone.java). + +```java +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.dns.Zone; +import com.google.gcloud.dns.ZoneInfo; + +Dns dns = DnsOptions.defaultInstance().service(); +String zoneName = "my-unique-zone"; +String domainName = "someexampledomain.com."; +String description = "This is a gcloud-java-dns sample zone."; +ZoneInfo zoneInfo = ZoneInfo.of(zoneName, domainName, description); +Zone zone = dns.create(zoneInfo); +``` + +The second snippet shows how to create records inside a zone. The complete code can be found on [CreateOrUpdateRecordSets.java](./gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateOrUpdateRecordSets.java). + +```java +import com.google.gcloud.dns.ChangeRequestInfo; +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.dns.RecordSet; +import com.google.gcloud.dns.Zone; + +import java.util.Iterator; +import java.util.concurrent.TimeUnit; + +Dns dns = DnsOptions.defaultInstance().service(); +String zoneName = "my-unique-zone"; +Zone zone = dns.getZone(zoneName); +String ip = "12.13.14.15"; +RecordSet toCreate = RecordSet.builder("www.someexampledomain.com.", RecordSet.Type.A) + .ttl(24, TimeUnit.HOURS) + .addRecord(ip) + .build(); +ChangeRequestInfo.Builder changeBuilder = ChangeRequestInfo.builder().add(toCreate); + +// Verify that the record does not exist yet. +// If it does exist, we will overwrite it with our prepared record. +Iterator recordSetIterator = zone.listRecordSets().iterateAll(); +while (recordSetIterator.hasNext()) { + RecordSet current = recordSetIterator.next(); + if (toCreate.name().equals(current.name()) && + toCreate.type().equals(current.type())) { + changeBuilder.delete(current); + } +} + +ChangeRequestInfo changeRequest = changeBuilder.build(); +zone.applyChangeRequest(changeRequest); +``` + Google Cloud Resource Manager (Alpha) ---------------------- @@ -358,6 +427,10 @@ Apache 2.0 - See [LICENSE] for more information. [cloud-datastore-activation]: https://cloud.google.com/datastore/docs/activate [datastore-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/datastore/package-summary.html +[dns-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/dns/package-summary.html +[cloud-dns-docs]: https://cloud.google.com/dns/docs +[cloud-dns-activation]: https://console.cloud.google.com/start/api?id=dns + [cloud-pubsub]: https://cloud.google.com/pubsub/ [cloud-pubsub-docs]: https://cloud.google.com/pubsub/docs diff --git a/codacy-conf.json b/codacy-conf.json index 61328436cba3..e8c819684c9c 100644 --- a/codacy-conf.json +++ b/codacy-conf.json @@ -1 +1 @@ -{"patterns":[{"patternId":"Custom_Javascript_Scopes","enabled":true},{"patternId":"Custom_Javascript_EvalWith","enabled":true},{"patternId":"Custom_Javascript_TryCatch","enabled":true},{"patternId":"Custom_Scala_NonFatal","enabled":true},{"patternId":"bitwise","enabled":true},{"patternId":"maxparams","enabled":true},{"patternId":"CSSLint_universal_selector","enabled":true},{"patternId":"CSSLint_unqualified_attributes","enabled":true},{"patternId":"CSSLint_zero_units","enabled":true},{"patternId":"CSSLint_overqualified_elements","enabled":true},{"patternId":"CSSLint_shorthand","enabled":true},{"patternId":"CSSLint_duplicate_background_images","enabled":true},{"patternId":"CSSLint_box_model","enabled":true},{"patternId":"CSSLint_compatible_vendor_prefixes","enabled":true},{"patternId":"CSSLint_display_property_grouping","enabled":true},{"patternId":"CSSLint_duplicate_properties","enabled":true},{"patternId":"CSSLint_empty_rules","enabled":true},{"patternId":"CSSLint_errors","enabled":true},{"patternId":"CSSLint_gradients","enabled":true},{"patternId":"CSSLint_important","enabled":true},{"patternId":"CSSLint_known_properties","enabled":true},{"patternId":"CSSLint_text_indent","enabled":true},{"patternId":"CSSLint_unique_headings","enabled":true},{"patternId":"PyLint_E0100","enabled":true},{"patternId":"PyLint_E0101","enabled":true},{"patternId":"PyLint_E0102","enabled":true},{"patternId":"PyLint_E0103","enabled":true},{"patternId":"PyLint_E0104","enabled":true},{"patternId":"PyLint_E0105","enabled":true},{"patternId":"PyLint_E0106","enabled":true},{"patternId":"PyLint_E0107","enabled":true},{"patternId":"PyLint_E0108","enabled":true},{"patternId":"PyLint_E0202","enabled":true},{"patternId":"PyLint_E0203","enabled":true},{"patternId":"PyLint_E0211","enabled":true},{"patternId":"PyLint_E0601","enabled":true},{"patternId":"PyLint_E0603","enabled":true},{"patternId":"PyLint_E0604","enabled":true},{"patternId":"PyLint_E0701","enabled":true},{"patternId":"PyLint_E0702","enabled":true},{"patternId":"PyLint_E0710","enabled":true},{"patternId":"PyLint_E0711","enabled":true},{"patternId":"PyLint_E0712","enabled":true},{"patternId":"PyLint_E1003","enabled":true},{"patternId":"PyLint_E1102","enabled":true},{"patternId":"PyLint_E1111","enabled":true},{"patternId":"PyLint_E1120","enabled":true},{"patternId":"PyLint_E1121","enabled":true},{"patternId":"PyLint_E1123","enabled":true},{"patternId":"PyLint_E1124","enabled":true},{"patternId":"PyLint_E1200","enabled":true},{"patternId":"PyLint_E1201","enabled":true},{"patternId":"PyLint_E1205","enabled":true},{"patternId":"PyLint_E1206","enabled":true},{"patternId":"PyLint_E1300","enabled":true},{"patternId":"PyLint_E1301","enabled":true},{"patternId":"PyLint_E1302","enabled":true},{"patternId":"PyLint_E1303","enabled":true},{"patternId":"PyLint_E1304","enabled":true},{"patternId":"PyLint_E1305","enabled":true},{"patternId":"PyLint_E1306","enabled":true},{"patternId":"rulesets-codesize.xml-CyclomaticComplexity","enabled":true},{"patternId":"rulesets-codesize.xml-NPathComplexity","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessiveMethodLength","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessiveClassLength","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessiveParameterList","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessivePublicCount","enabled":true},{"patternId":"rulesets-codesize.xml-TooManyFields","enabled":true},{"patternId":"rulesets-codesize.xml-TooManyMethods","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessiveClassComplexity","enabled":true},{"patternId":"rulesets-controversial.xml-Superglobals","enabled":true},{"patternId":"rulesets-design.xml-ExitExpression","enabled":true},{"patternId":"rulesets-design.xml-EvalExpression","enabled":true},{"patternId":"rulesets-design.xml-GotoStatement","enabled":true},{"patternId":"rulesets-design.xml-NumberOfChildren","enabled":true},{"patternId":"rulesets-design.xml-DepthOfInheritance","enabled":true},{"patternId":"rulesets-unusedcode.xml-UnusedPrivateField","enabled":true},{"patternId":"rulesets-unusedcode.xml-UnusedLocalVariable","enabled":true},{"patternId":"rulesets-unusedcode.xml-UnusedPrivateMethod","enabled":true},{"patternId":"rulesets-unusedcode.xml-UnusedFormalParameter","enabled":true},{"patternId":"PyLint_C0303","enabled":true},{"patternId":"PyLint_C1001","enabled":true},{"patternId":"rulesets-naming.xml-ShortVariable","enabled":true},{"patternId":"rulesets-naming.xml-LongVariable","enabled":true},{"patternId":"rulesets-naming.xml-ShortMethodName","enabled":true},{"patternId":"rulesets-naming.xml-ConstantNamingConventions","enabled":true},{"patternId":"rulesets-naming.xml-BooleanGetMethodName","enabled":true},{"patternId":"PyLint_W0101","enabled":true},{"patternId":"PyLint_W0102","enabled":true},{"patternId":"PyLint_W0104","enabled":true},{"patternId":"PyLint_W0105","enabled":true},{"patternId":"Custom_Scala_GetCalls","enabled":true},{"patternId":"ScalaStyle_EqualsHashCodeChecker","enabled":true},{"patternId":"ScalaStyle_ParameterNumberChecker","enabled":true},{"patternId":"ScalaStyle_ReturnChecker","enabled":true},{"patternId":"ScalaStyle_NullChecker","enabled":true},{"patternId":"ScalaStyle_NoCloneChecker","enabled":true},{"patternId":"ScalaStyle_NoFinalizeChecker","enabled":true},{"patternId":"ScalaStyle_CovariantEqualsChecker","enabled":true},{"patternId":"ScalaStyle_StructuralTypeChecker","enabled":true},{"patternId":"ScalaStyle_MethodLengthChecker","enabled":true},{"patternId":"ScalaStyle_NumberOfMethodsInTypeChecker","enabled":true},{"patternId":"ScalaStyle_WhileChecker","enabled":true},{"patternId":"ScalaStyle_VarFieldChecker","enabled":true},{"patternId":"ScalaStyle_VarLocalChecker","enabled":true},{"patternId":"ScalaStyle_RedundantIfChecker","enabled":true},{"patternId":"ScalaStyle_DeprecatedJavaChecker","enabled":true},{"patternId":"ScalaStyle_EmptyClassChecker","enabled":true},{"patternId":"ScalaStyle_NotImplementedErrorUsage","enabled":true},{"patternId":"Custom_Scala_GroupImports","enabled":true},{"patternId":"Custom_Scala_ReservedKeywords","enabled":true},{"patternId":"Custom_Scala_ElseIf","enabled":true},{"patternId":"Custom_Scala_CallByNameAsLastArguments","enabled":true},{"patternId":"Custom_Scala_WildcardImportOnMany","enabled":true},{"patternId":"Custom_Scala_UtilTryForTryCatch","enabled":true},{"patternId":"Custom_Scala_ProhibitObjectName","enabled":true},{"patternId":"Custom_Scala_ImportsAtBeginningOfPackage","enabled":true},{"patternId":"Custom_Scala_NameResultsAndParameters","enabled":true},{"patternId":"Custom_Scala_IncompletePatternMatching","enabled":true},{"patternId":"Custom_Scala_UsefulTypeAlias","enabled":true},{"patternId":"Custom_Scala_JavaThreads","enabled":true},{"patternId":"Custom_Scala_DirectPromiseCreation","enabled":true},{"patternId":"Custom_Scala_StructuralTypes","enabled":true},{"patternId":"Custom_Scala_CollectionLastHead","enabled":true},{"patternId":"PyLint_W0106","enabled":true},{"patternId":"PyLint_W0107","enabled":true},{"patternId":"PyLint_W0108","enabled":true},{"patternId":"PyLint_W0109","enabled":true},{"patternId":"PyLint_W0110","enabled":true},{"patternId":"PyLint_W0120","enabled":true},{"patternId":"PyLint_W0122","enabled":true},{"patternId":"PyLint_W0150","enabled":true},{"patternId":"PyLint_W0199","enabled":true},{"patternId":"rulesets-cleancode.xml-ElseExpression","enabled":true},{"patternId":"rulesets-cleancode.xml-StaticAccess","enabled":true},{"patternId":"ScalaStyle_NonASCIICharacterChecker","enabled":true},{"patternId":"ScalaStyle_FieldNamesChecker","enabled":true},{"patternId":"Custom_Scala_WithNameCalls","enabled":true},{"patternId":"braces_IfElseStmtsMustUseBraces","enabled":true},{"patternId":"basic_AvoidDecimalLiteralsInBigDecimalConstructor","enabled":true},{"patternId":"basic_CheckSkipResult","enabled":true},{"patternId":"design_AvoidInstanceofChecksInCatchClause","enabled":true},{"patternId":"j2ee_DoNotCallSystemExit","enabled":true},{"patternId":"unusedcode_UnusedLocalVariable","enabled":true},{"patternId":"basic_DontUseFloatTypeForLoopIndices","enabled":true},{"patternId":"basic_AvoidBranchingStatementAsLastInLoop","enabled":true},{"patternId":"empty_EmptyFinallyBlock","enabled":true},{"patternId":"design_CompareObjectsWithEquals","enabled":true},{"patternId":"junit_UnnecessaryBooleanAssertion","enabled":true},{"patternId":"design_SimplifyBooleanExpressions","enabled":true},{"patternId":"basic_JumbledIncrementer","enabled":true},{"patternId":"design_SwitchStmtsShouldHaveDefault","enabled":true},{"patternId":"strictexception_AvoidThrowingRawExceptionTypes","enabled":true},{"patternId":"design_SimplifyBooleanReturns","enabled":true},{"patternId":"empty_EmptyInitializer","enabled":true},{"patternId":"design_FieldDeclarationsShouldBeAtStartOfClass","enabled":true},{"patternId":"naming_PackageCase","enabled":true},{"patternId":"controversial_UnnecessaryConstructor","enabled":true},{"patternId":"naming_MethodNamingConventions","enabled":true},{"patternId":"basic_UnconditionalIfStatement","enabled":true},{"patternId":"design_SingularField","enabled":true},{"patternId":"design_AssignmentToNonFinalStatic","enabled":true},{"patternId":"strings_UseStringBufferLength","enabled":true},{"patternId":"finalizers_AvoidCallingFinalize","enabled":true},{"patternId":"naming_ClassNamingConventions","enabled":true},{"patternId":"imports_UnnecessaryFullyQualifiedName","enabled":true},{"patternId":"unusedcode_UnusedPrivateField","enabled":true},{"patternId":"strings_UnnecessaryCaseChange","enabled":true},{"patternId":"design_NonStaticInitializer","enabled":true},{"patternId":"design_MissingBreakInSwitch","enabled":true},{"patternId":"design_AvoidReassigningParameters","enabled":true},{"patternId":"basic_AvoidThreadGroup","enabled":true},{"patternId":"codesize_ExcessiveParameterList","parameters":{"minimum":"8","violationSuppressRegex":"\"\"","violationSuppressXPath":"\"\""},"enabled":true},{"patternId":"design_UncommentedEmptyMethodBody","enabled":true},{"patternId":"basic_BrokenNullCheck","enabled":true},{"patternId":"strings_StringInstantiation","enabled":true},{"patternId":"design_EqualsNull","enabled":true},{"patternId":"basic_OverrideBothEqualsAndHashcode","enabled":true},{"patternId":"basic_BooleanInstantiation","enabled":true},{"patternId":"basic_ReturnFromFinallyBlock","enabled":true},{"patternId":"empty_EmptyTryBlock","enabled":true},{"patternId":"basic_ExtendsObject","enabled":true},{"patternId":"strictexception_AvoidThrowingNullPointerException","enabled":true},{"patternId":"empty_EmptySwitchStatements","enabled":true},{"patternId":"basic_MisplacedNullCheck","enabled":true},{"patternId":"strings_StringToString","enabled":true},{"patternId":"naming_MethodWithSameNameAsEnclosingClass","enabled":true},{"patternId":"imports_UnusedImports","enabled":true},{"patternId":"basic_AvoidMultipleUnaryOperators","enabled":true},{"patternId":"junit_SimplifyBooleanAssertion","enabled":true},{"patternId":"braces_IfStmtsMustUseBraces","enabled":true},{"patternId":"naming_NoPackage","enabled":true},{"patternId":"unusedcode_UnusedFormalParameter","enabled":true},{"patternId":"empty_EmptyStatementNotInLoop","enabled":true},{"patternId":"naming_GenericsNaming","enabled":true},{"patternId":"strings_UseEqualsToCompareStrings","enabled":true},{"patternId":"empty_EmptyStaticInitializer","enabled":true},{"patternId":"empty_EmptyStatementBlock","enabled":true},{"patternId":"basic_CollapsibleIfStatements","enabled":true},{"patternId":"design_ImmutableField","enabled":true},{"patternId":"controversial_OneDeclarationPerLine","enabled":true},{"patternId":"unnecessary_UnnecessaryReturn","enabled":true},{"patternId":"codesize_NPathComplexity","enabled":true},{"patternId":"imports_DontImportJavaLang","enabled":true},{"patternId":"empty_EmptySynchronizedBlock","enabled":true},{"patternId":"unnecessary_UselessOperationOnImmutable","enabled":true},{"patternId":"design_PositionLiteralsFirstInComparisons","enabled":true},{"patternId":"junit_JUnitSpelling","enabled":true},{"patternId":"finalizers_EmptyFinalizer","enabled":true},{"patternId":"design_NonCaseLabelInSwitchStatement","enabled":true},{"patternId":"android_DoNotHardCodeSDCard","enabled":true},{"patternId":"design_LogicInversion","enabled":true},{"patternId":"unusedcode_UnusedPrivateMethod","enabled":true},{"patternId":"basic_CheckResultSet","enabled":true},{"patternId":"controversial_AvoidPrefixingMethodParameters","enabled":true},{"patternId":"empty_EmptyIfStmt","enabled":true},{"patternId":"basic_DontCallThreadRun","enabled":true},{"patternId":"junit_JUnitStaticSuite","enabled":true},{"patternId":"codesize_ExcessiveMethodLength","enabled":true},{"patternId":"design_MissingStaticMethodInNonInstantiatableClass","enabled":true},{"patternId":"Style_MethodName","enabled":true},{"patternId":"Metrics_CyclomaticComplexity","enabled":true},{"patternId":"Lint_DuplicateMethods","enabled":true},{"patternId":"Style_Lambda","enabled":true},{"patternId":"Lint_UselessSetterCall","enabled":true},{"patternId":"Style_VariableName","enabled":true},{"patternId":"Lint_AmbiguousOperator","enabled":true},{"patternId":"Style_LeadingCommentSpace","enabled":true},{"patternId":"Style_CaseEquality","enabled":true},{"patternId":"Lint_StringConversionInInterpolation","enabled":true},{"patternId":"Performance_ReverseEach","enabled":true},{"patternId":"Lint_LiteralInCondition","enabled":true},{"patternId":"Performance_Sample","enabled":true},{"patternId":"Style_NonNilCheck","enabled":true},{"patternId":"Lint_RescueException","enabled":true},{"patternId":"Lint_UselessElseWithoutRescue","enabled":true},{"patternId":"Style_ConstantName","enabled":true},{"patternId":"Lint_LiteralInInterpolation","enabled":true},{"patternId":"Lint_NestedMethodDefinition","enabled":true},{"patternId":"Style_DoubleNegation","enabled":true},{"patternId":"Lint_SpaceBeforeFirstArg","enabled":true},{"patternId":"Lint_Debugger","enabled":true},{"patternId":"Style_ClassVars","enabled":true},{"patternId":"Lint_EmptyEnsure","enabled":true},{"patternId":"Style_MultilineBlockLayout","enabled":true},{"patternId":"Lint_UnusedBlockArgument","enabled":true},{"patternId":"Lint_UselessAccessModifier","enabled":true},{"patternId":"Performance_Size","enabled":true},{"patternId":"Lint_EachWithObjectArgument","enabled":true},{"patternId":"Style_Alias","enabled":true},{"patternId":"Lint_Loop","enabled":true},{"patternId":"Style_NegatedWhile","enabled":true},{"patternId":"Style_ColonMethodCall","enabled":true},{"patternId":"Lint_AmbiguousRegexpLiteral","enabled":true},{"patternId":"Lint_UnusedMethodArgument","enabled":true},{"patternId":"Style_MultilineIfThen","enabled":true},{"patternId":"Lint_EnsureReturn","enabled":true},{"patternId":"Style_NegatedIf","enabled":true},{"patternId":"Lint_Eval","enabled":true},{"patternId":"Style_NilComparison","enabled":true},{"patternId":"Style_ArrayJoin","enabled":true},{"patternId":"Lint_ConditionPosition","enabled":true},{"patternId":"Lint_UnreachableCode","enabled":true},{"patternId":"Performance_Count","enabled":true},{"patternId":"Lint_EmptyInterpolation","enabled":true},{"patternId":"Style_LambdaCall","enabled":true},{"patternId":"Lint_HandleExceptions","enabled":true},{"patternId":"Lint_ShadowingOuterLocalVariable","enabled":true},{"patternId":"Lint_EndAlignment","enabled":true},{"patternId":"Style_MultilineTernaryOperator","enabled":true},{"patternId":"Style_AutoResourceCleanup","enabled":true},{"patternId":"Lint_ElseLayout","enabled":true},{"patternId":"Style_NestedTernaryOperator","enabled":true},{"patternId":"Style_OneLineConditional","enabled":true},{"patternId":"Style_EmptyElse","enabled":true},{"patternId":"Lint_UselessComparison","enabled":true},{"patternId":"Metrics_PerceivedComplexity","enabled":true},{"patternId":"Style_InfiniteLoop","enabled":true},{"patternId":"Rails_Date","enabled":true},{"patternId":"Style_EvenOdd","enabled":true},{"patternId":"Style_IndentationConsistency","enabled":true},{"patternId":"Style_ModuleFunction","enabled":true},{"patternId":"Lint_UselessAssignment","enabled":true},{"patternId":"Style_EachWithObject","enabled":true},{"patternId":"Performance_Detect","enabled":true},{"patternId":"duplicate_key","enabled":true},{"patternId":"no_interpolation_in_single_quotes","enabled":true},{"patternId":"no_backticks","enabled":true},{"patternId":"no_unnecessary_fat_arrows","enabled":true},{"patternId":"indentation","enabled":true},{"patternId":"ensure_comprehensions","enabled":true},{"patternId":"no_stand_alone_at","enabled":true},{"patternId":"cyclomatic_complexity","enabled":true},{"patternId":"Deserialize","enabled":true},{"patternId":"SymbolDoS","enabled":true},{"patternId":"SkipBeforeFilter","enabled":true},{"patternId":"SanitizeMethods","enabled":true},{"patternId":"SelectTag","enabled":true},{"patternId":"XMLDoS","enabled":true},{"patternId":"SimpleFormat","enabled":true},{"patternId":"Evaluation","enabled":true},{"patternId":"BasicAuth","enabled":true},{"patternId":"JRubyXML","enabled":true},{"patternId":"RenderInline","enabled":true},{"patternId":"YAMLParsing","enabled":true},{"patternId":"Redirect","enabled":true},{"patternId":"UnsafeReflection","enabled":true},{"patternId":"SSLVerify","enabled":true},{"patternId":"HeaderDoS","enabled":true},{"patternId":"TranslateBug","enabled":true},{"patternId":"Execute","enabled":true},{"patternId":"JSONParsing","enabled":true},{"patternId":"LinkTo","enabled":true},{"patternId":"FileDisclosure","enabled":true},{"patternId":"SafeBufferManipulation","enabled":true},{"patternId":"ModelAttributes","enabled":true},{"patternId":"ResponseSplitting","enabled":true},{"patternId":"DigestDoS","enabled":true},{"patternId":"Send","enabled":true},{"patternId":"MailTo","enabled":true},{"patternId":"SymbolDoSCVE","enabled":true},{"patternId":"StripTags","enabled":true},{"patternId":"MassAssignment","enabled":true},{"patternId":"RegexDoS","enabled":true},{"patternId":"SelectVulnerability","enabled":true},{"patternId":"FileAccess","enabled":true},{"patternId":"ContentTag","enabled":true},{"patternId":"SessionSettings","enabled":true},{"patternId":"FilterSkipping","enabled":true},{"patternId":"CreateWith","enabled":true},{"patternId":"JSONEncoding","enabled":true},{"patternId":"SQLCVEs","enabled":true},{"patternId":"ForgerySetting","enabled":true},{"patternId":"QuoteTableName","enabled":true},{"patternId":"I18nXSS","enabled":true},{"patternId":"WithoutProtection","enabled":true},{"patternId":"CrossSiteScripting","enabled":true},{"patternId":"SingleQuotes","enabled":true},{"patternId":"NestedAttributes","enabled":true},{"patternId":"DetailedExceptions","enabled":true},{"patternId":"LinkToHref","enabled":true},{"patternId":"RenderDoS","enabled":true},{"patternId":"ModelSerialize","enabled":true},{"patternId":"SQL","enabled":true},{"patternId":"Render","enabled":true},{"patternId":"UnscopedFind","enabled":true},{"patternId":"ValidationRegex","enabled":true},{"patternId":"EscapeFunction","enabled":true},{"patternId":"Custom_Scala_FieldNamesChecker","enabled":true},{"patternId":"Custom_Scala_ObjDeserialization","enabled":true},{"patternId":"Custom_Scala_RSAPadding","enabled":true},{"patternId":"ESLint_no-extra-boolean-cast","enabled":true},{"patternId":"ESLint_no-iterator","enabled":true},{"patternId":"ESLint_no-invalid-regexp","enabled":true},{"patternId":"ESLint_no-obj-calls","enabled":true},{"patternId":"ESLint_no-sparse-arrays","enabled":true},{"patternId":"ESLint_no-unreachable","enabled":true},{"patternId":"ESLint_no-dupe-keys","enabled":true},{"patternId":"ESLint_no-multi-str","enabled":true},{"patternId":"ESLint_no-extend-native","enabled":true},{"patternId":"ESLint_guard-for-in","enabled":true},{"patternId":"ESLint_no-func-assign","enabled":true},{"patternId":"ESLint_no-extra-semi","enabled":true},{"patternId":"ESLint_camelcase","enabled":true},{"patternId":"ESLint_no-mixed-spaces-and-tabs","enabled":true},{"patternId":"ESLint_no-undef","enabled":true},{"patternId":"ESLint_semi","enabled":true},{"patternId":"ESLint_no-empty-character-class","enabled":true},{"patternId":"ESLint_complexity","enabled":true},{"patternId":"ESLint_no-dupe-class-members","enabled":true},{"patternId":"ESLint_no-debugger","enabled":true},{"patternId":"ESLint_block-scoped-var","enabled":true},{"patternId":"ESLint_no-loop-func","enabled":true},{"patternId":"ESLint_no-use-before-define","enabled":true},{"patternId":"ESLint_no-console","enabled":true},{"patternId":"ESLint_require-yield","enabled":true},{"patternId":"ESLint_no-redeclare","enabled":true},{"patternId":"ESLint_no-undefined","enabled":true},{"patternId":"ESLint_use-isnan","enabled":true},{"patternId":"ESLint_no-control-regex","enabled":true},{"patternId":"ESLint_no-const-assign","enabled":true},{"patternId":"ESLint_no-new","enabled":true},{"patternId":"ESLint_new-cap","enabled":true},{"patternId":"ESLint_no-irregular-whitespace","enabled":true},{"patternId":"ESLint_object-shorthand","enabled":true},{"patternId":"ESLint_no-ex-assign","enabled":true},{"patternId":"ESLint_wrap-iife","enabled":true},{"patternId":"ESLint_arrow-parens","enabled":true},{"patternId":"ESLint_no-constant-condition","enabled":true},{"patternId":"ESLint_no-octal","enabled":true},{"patternId":"ESLint_no-dupe-args","enabled":true},{"patternId":"ESLint_quotes","enabled":true},{"patternId":"ESLint_no-fallthrough","enabled":true},{"patternId":"ESLint_no-delete-var","enabled":true},{"patternId":"ESLint_no-caller","enabled":true},{"patternId":"ESLint_no-cond-assign","enabled":true},{"patternId":"ESLint_no-this-before-super","enabled":true},{"patternId":"ESLint_no-negated-in-lhs","enabled":true},{"patternId":"ESLint_no-inner-declarations","enabled":true},{"patternId":"ESLint_eqeqeq","enabled":true},{"patternId":"ESLint_curly","enabled":true},{"patternId":"ESLint_arrow-spacing","enabled":true},{"patternId":"ESLint_no-empty","enabled":true},{"patternId":"ESLint_no-unused-vars","enabled":true},{"patternId":"ESLint_generator-star-spacing","enabled":true},{"patternId":"ESLint_no-duplicate-case","enabled":true},{"patternId":"ESLint_valid-typeof","enabled":true},{"patternId":"ESLint_no-regex-spaces","enabled":true},{"patternId":"ESLint_no-class-assign","enabled":true},{"patternId":"PyLint_W0221","enabled":true},{"patternId":"PyLint_E0117","enabled":true},{"patternId":"PyLint_E0001","enabled":true},{"patternId":"PyLint_E0241","enabled":true},{"patternId":"PyLint_W0404","enabled":true},{"patternId":"PyLint_E0704","enabled":true},{"patternId":"PyLint_E0703","enabled":true},{"patternId":"PyLint_E0302","enabled":true},{"patternId":"PyLint_W1301","enabled":true},{"patternId":"PyLint_R0201","enabled":true},{"patternId":"PyLint_E0113","enabled":true},{"patternId":"PyLint_W0410","enabled":true},{"patternId":"PyLint_C0123","enabled":true},{"patternId":"PyLint_E0115","enabled":true},{"patternId":"PyLint_E0114","enabled":true},{"patternId":"PyLint_E1126","enabled":true},{"patternId":"PyLint_W0702","enabled":true},{"patternId":"PyLint_W1303","enabled":true},{"patternId":"PyLint_W0622","enabled":true},{"patternId":"PyLint_W0222","enabled":true},{"patternId":"PyLint_W0233","enabled":true},{"patternId":"PyLint_W1305","enabled":true},{"patternId":"PyLint_E1127","enabled":true},{"patternId":"PyLint_E0112","enabled":true},{"patternId":"PyLint_W0611","enabled":true},{"patternId":"PyLint_W0601","enabled":true},{"patternId":"PyLint_W1300","enabled":true},{"patternId":"PyLint_W0124","enabled":true},{"patternId":"PyLint_R0203","enabled":true},{"patternId":"PyLint_E0236","enabled":true},{"patternId":"PyLint_W0612","enabled":true},{"patternId":"PyLint_W0604","enabled":true},{"patternId":"PyLint_W0705","enabled":true},{"patternId":"PyLint_E0238","enabled":true},{"patternId":"PyLint_W0602","enabled":true},{"patternId":"PyLint_R0102","enabled":true},{"patternId":"PyLint_R0202","enabled":true},{"patternId":"PyLint_E0240","enabled":true},{"patternId":"PyLint_W0623","enabled":true},{"patternId":"PyLint_W0711","enabled":true},{"patternId":"PyLint_E0116","enabled":true},{"patternId":"PyLint_E0239","enabled":true},{"patternId":"PyLint_E1132","enabled":true},{"patternId":"PyLint_W1307","enabled":true},{"patternId":"PyLint_C0200","enabled":true},{"patternId":"PyLint_E0301","enabled":true},{"patternId":"PyLint_W1306","enabled":true},{"patternId":"PyLint_W1302","enabled":true},{"patternId":"PyLint_E0110","enabled":true},{"patternId":"PyLint_E1125","enabled":true}]} \ No newline at end of file +{"patterns":[{"patternId":"Custom_Javascript_Scopes","enabled":true},{"patternId":"Custom_Javascript_EvalWith","enabled":true},{"patternId":"Custom_Javascript_TryCatch","enabled":true},{"patternId":"Custom_Scala_NonFatal","enabled":true},{"patternId":"bitwise","enabled":true},{"patternId":"maxparams","enabled":true},{"patternId":"CSSLint_universal_selector","enabled":true},{"patternId":"CSSLint_unqualified_attributes","enabled":true},{"patternId":"CSSLint_zero_units","enabled":true},{"patternId":"CSSLint_overqualified_elements","enabled":true},{"patternId":"CSSLint_shorthand","enabled":true},{"patternId":"CSSLint_duplicate_background_images","enabled":true},{"patternId":"CSSLint_box_model","enabled":true},{"patternId":"CSSLint_compatible_vendor_prefixes","enabled":true},{"patternId":"CSSLint_display_property_grouping","enabled":true},{"patternId":"CSSLint_duplicate_properties","enabled":true},{"patternId":"CSSLint_empty_rules","enabled":true},{"patternId":"CSSLint_errors","enabled":true},{"patternId":"CSSLint_gradients","enabled":true},{"patternId":"CSSLint_important","enabled":true},{"patternId":"CSSLint_known_properties","enabled":true},{"patternId":"CSSLint_text_indent","enabled":true},{"patternId":"CSSLint_unique_headings","enabled":true},{"patternId":"PyLint_E0100","enabled":true},{"patternId":"PyLint_E0101","enabled":true},{"patternId":"PyLint_E0102","enabled":true},{"patternId":"PyLint_E0103","enabled":true},{"patternId":"PyLint_E0104","enabled":true},{"patternId":"PyLint_E0105","enabled":true},{"patternId":"PyLint_E0106","enabled":true},{"patternId":"PyLint_E0107","enabled":true},{"patternId":"PyLint_E0108","enabled":true},{"patternId":"PyLint_E0202","enabled":true},{"patternId":"PyLint_E0203","enabled":true},{"patternId":"PyLint_E0211","enabled":true},{"patternId":"PyLint_E0601","enabled":true},{"patternId":"PyLint_E0603","enabled":true},{"patternId":"PyLint_E0604","enabled":true},{"patternId":"PyLint_E0701","enabled":true},{"patternId":"PyLint_E0702","enabled":true},{"patternId":"PyLint_E0710","enabled":true},{"patternId":"PyLint_E0711","enabled":true},{"patternId":"PyLint_E0712","enabled":true},{"patternId":"PyLint_E1003","enabled":true},{"patternId":"PyLint_E1102","enabled":true},{"patternId":"PyLint_E1111","enabled":true},{"patternId":"PyLint_E1120","enabled":true},{"patternId":"PyLint_E1121","enabled":true},{"patternId":"PyLint_E1123","enabled":true},{"patternId":"PyLint_E1124","enabled":true},{"patternId":"PyLint_E1200","enabled":true},{"patternId":"PyLint_E1201","enabled":true},{"patternId":"PyLint_E1205","enabled":true},{"patternId":"PyLint_E1206","enabled":true},{"patternId":"PyLint_E1300","enabled":true},{"patternId":"PyLint_E1301","enabled":true},{"patternId":"PyLint_E1302","enabled":true},{"patternId":"PyLint_E1303","enabled":true},{"patternId":"PyLint_E1304","enabled":true},{"patternId":"PyLint_E1305","enabled":true},{"patternId":"PyLint_E1306","enabled":true},{"patternId":"rulesets-codesize.xml-CyclomaticComplexity","enabled":true},{"patternId":"rulesets-codesize.xml-NPathComplexity","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessiveMethodLength","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessiveClassLength","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessiveParameterList","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessivePublicCount","enabled":true},{"patternId":"rulesets-codesize.xml-TooManyFields","enabled":true},{"patternId":"rulesets-codesize.xml-TooManyMethods","enabled":true},{"patternId":"rulesets-codesize.xml-ExcessiveClassComplexity","enabled":true},{"patternId":"rulesets-controversial.xml-Superglobals","enabled":true},{"patternId":"rulesets-design.xml-ExitExpression","enabled":true},{"patternId":"rulesets-design.xml-EvalExpression","enabled":true},{"patternId":"rulesets-design.xml-GotoStatement","enabled":true},{"patternId":"rulesets-design.xml-NumberOfChildren","enabled":true},{"patternId":"rulesets-design.xml-DepthOfInheritance","enabled":true},{"patternId":"rulesets-unusedcode.xml-UnusedPrivateField","enabled":true},{"patternId":"rulesets-unusedcode.xml-UnusedLocalVariable","enabled":true},{"patternId":"rulesets-unusedcode.xml-UnusedPrivateMethod","enabled":true},{"patternId":"rulesets-unusedcode.xml-UnusedFormalParameter","enabled":true},{"patternId":"PyLint_C0303","enabled":true},{"patternId":"PyLint_C1001","enabled":true},{"patternId":"rulesets-naming.xml-ShortVariable","enabled":true},{"patternId":"rulesets-naming.xml-LongVariable","enabled":true},{"patternId":"rulesets-naming.xml-ShortMethodName","enabled":true},{"patternId":"rulesets-naming.xml-ConstantNamingConventions","enabled":true},{"patternId":"rulesets-naming.xml-BooleanGetMethodName","enabled":true},{"patternId":"PyLint_W0101","enabled":true},{"patternId":"PyLint_W0102","enabled":true},{"patternId":"PyLint_W0104","enabled":true},{"patternId":"PyLint_W0105","enabled":true},{"patternId":"Custom_Scala_GetCalls","enabled":true},{"patternId":"ScalaStyle_EqualsHashCodeChecker","enabled":true},{"patternId":"ScalaStyle_ParameterNumberChecker","enabled":true},{"patternId":"ScalaStyle_ReturnChecker","enabled":true},{"patternId":"ScalaStyle_NullChecker","enabled":true},{"patternId":"ScalaStyle_NoCloneChecker","enabled":true},{"patternId":"ScalaStyle_NoFinalizeChecker","enabled":true},{"patternId":"ScalaStyle_CovariantEqualsChecker","enabled":true},{"patternId":"ScalaStyle_StructuralTypeChecker","enabled":true},{"patternId":"ScalaStyle_MethodLengthChecker","enabled":true},{"patternId":"ScalaStyle_NumberOfMethodsInTypeChecker","enabled":true},{"patternId":"ScalaStyle_WhileChecker","enabled":true},{"patternId":"ScalaStyle_VarFieldChecker","enabled":true},{"patternId":"ScalaStyle_VarLocalChecker","enabled":true},{"patternId":"ScalaStyle_RedundantIfChecker","enabled":true},{"patternId":"ScalaStyle_DeprecatedJavaChecker","enabled":true},{"patternId":"ScalaStyle_EmptyClassChecker","enabled":true},{"patternId":"ScalaStyle_NotImplementedErrorUsage","enabled":true},{"patternId":"Custom_Scala_GroupImports","enabled":true},{"patternId":"Custom_Scala_ReservedKeywords","enabled":true},{"patternId":"Custom_Scala_ElseIf","enabled":true},{"patternId":"Custom_Scala_CallByNameAsLastArguments","enabled":true},{"patternId":"Custom_Scala_WildcardImportOnMany","enabled":true},{"patternId":"Custom_Scala_UtilTryForTryCatch","enabled":true},{"patternId":"Custom_Scala_ProhibitObjectName","enabled":true},{"patternId":"Custom_Scala_ImportsAtBeginningOfPackage","enabled":true},{"patternId":"Custom_Scala_NameResultsAndParameters","enabled":true},{"patternId":"Custom_Scala_IncompletePatternMatching","enabled":true},{"patternId":"Custom_Scala_UsefulTypeAlias","enabled":true},{"patternId":"Custom_Scala_JavaThreads","enabled":true},{"patternId":"Custom_Scala_DirectPromiseCreation","enabled":true},{"patternId":"Custom_Scala_StructuralTypes","enabled":true},{"patternId":"Custom_Scala_CollectionLastHead","enabled":true},{"patternId":"PyLint_W0106","enabled":true},{"patternId":"PyLint_W0107","enabled":true},{"patternId":"PyLint_W0108","enabled":true},{"patternId":"PyLint_W0109","enabled":true},{"patternId":"PyLint_W0110","enabled":true},{"patternId":"PyLint_W0120","enabled":true},{"patternId":"PyLint_W0122","enabled":true},{"patternId":"PyLint_W0150","enabled":true},{"patternId":"PyLint_W0199","enabled":true},{"patternId":"rulesets-cleancode.xml-ElseExpression","enabled":true},{"patternId":"rulesets-cleancode.xml-StaticAccess","enabled":true},{"patternId":"ScalaStyle_NonASCIICharacterChecker","enabled":true},{"patternId":"ScalaStyle_FieldNamesChecker","enabled":true},{"patternId":"Custom_Scala_WithNameCalls","enabled":true},{"patternId":"strictexception_AvoidRethrowingException","enabled":true},{"patternId":"strings_AppendCharacterWithChar","enabled":true},{"patternId":"braces_IfElseStmtsMustUseBraces","enabled":true},{"patternId":"basic_AvoidDecimalLiteralsInBigDecimalConstructor","enabled":true},{"patternId":"basic_CheckSkipResult","enabled":true},{"patternId":"javabeans_MissingSerialVersionUID","enabled":true},{"patternId":"migrating_ShortInstantiation","enabled":true},{"patternId":"design_AvoidInstanceofChecksInCatchClause","enabled":true},{"patternId":"naming_LongVariable","enabled":true},{"patternId":"migrating_ReplaceEnumerationWithIterator","enabled":true},{"patternId":"j2ee_DoNotCallSystemExit","enabled":true},{"patternId":"unusedcode_UnusedLocalVariable","enabled":true},{"patternId":"strings_InefficientStringBuffering","enabled":true},{"patternId":"basic_DontUseFloatTypeForLoopIndices","enabled":true},{"patternId":"basic_AvoidBranchingStatementAsLastInLoop","enabled":true},{"patternId":"migrating_JUnit4TestShouldUseTestAnnotation","enabled":true},{"patternId":"optimizations_AddEmptyString","enabled":true},{"patternId":"logging-jakarta-commons_ProperLogger","enabled":true},{"patternId":"optimizations_RedundantFieldInitializer","enabled":true},{"patternId":"logging-java_AvoidPrintStackTrace","enabled":true},{"patternId":"empty_EmptyFinallyBlock","enabled":true},{"patternId":"design_CompareObjectsWithEquals","enabled":true},{"patternId":"basic_ClassCastExceptionWithToArray","enabled":true},{"patternId":"strictexception_DoNotExtendJavaLangError","enabled":true},{"patternId":"junit_UnnecessaryBooleanAssertion","enabled":true},{"patternId":"design_SimplifyBooleanExpressions","enabled":true},{"patternId":"basic_ForLoopShouldBeWhileLoop","enabled":true},{"patternId":"basic_BigIntegerInstantiation","enabled":true},{"patternId":"optimizations_UseArrayListInsteadOfVector","enabled":true},{"patternId":"optimizations_UnnecessaryWrapperObjectCreation","enabled":true},{"patternId":"strings_StringBufferInstantiationWithChar","enabled":true},{"patternId":"basic_JumbledIncrementer","enabled":true},{"patternId":"design_SwitchStmtsShouldHaveDefault","enabled":true},{"patternId":"strictexception_AvoidThrowingRawExceptionTypes","enabled":true},{"patternId":"migrating_LongInstantiation","enabled":true},{"patternId":"design_SimplifyBooleanReturns","enabled":true},{"patternId":"empty_EmptyInitializer","enabled":true},{"patternId":"design_FieldDeclarationsShouldBeAtStartOfClass","enabled":true},{"patternId":"unnecessary_UnnecessaryConversionTemporary","enabled":true},{"patternId":"design_AvoidProtectedFieldInFinalClass","enabled":true},{"patternId":"junit_UseAssertTrueInsteadOfAssertEquals","enabled":true},{"patternId":"naming_PackageCase","enabled":true},{"patternId":"migrating_JUnitUseExpected","enabled":true},{"patternId":"controversial_UnnecessaryConstructor","enabled":true},{"patternId":"naming_MethodNamingConventions","enabled":true},{"patternId":"design_DefaultLabelNotLastInSwitchStmt","enabled":true},{"patternId":"basic_UnconditionalIfStatement","enabled":true},{"patternId":"design_SingularField","enabled":true},{"patternId":"design_AssignmentToNonFinalStatic","enabled":true},{"patternId":"braces_WhileLoopsMustUseBraces","enabled":true},{"patternId":"logging-java_SystemPrintln","enabled":true},{"patternId":"strings_UseStringBufferLength","enabled":true},{"patternId":"controversial_AvoidUsingNativeCode","enabled":true},{"patternId":"strictexception_AvoidLosingExceptionInformation","enabled":true},{"patternId":"imports_ImportFromSamePackage","enabled":true},{"patternId":"finalizers_AvoidCallingFinalize","enabled":true},{"patternId":"finalizers_FinalizeOverloaded","enabled":true},{"patternId":"naming_ClassNamingConventions","enabled":true},{"patternId":"logging-java_LoggerIsNotStaticFinal","enabled":true},{"patternId":"finalizers_FinalizeOnlyCallsSuperFinalize","enabled":true},{"patternId":"unnecessary_UselessOverridingMethod","enabled":true},{"patternId":"naming_SuspiciousConstantFieldName","enabled":true},{"patternId":"design_OptimizableToArrayCall","enabled":true},{"patternId":"imports_UnnecessaryFullyQualifiedName","enabled":true},{"patternId":"migrating_ReplaceHashtableWithMap","enabled":true},{"patternId":"unusedcode_UnusedPrivateField","enabled":true},{"patternId":"strings_UnnecessaryCaseChange","enabled":true},{"patternId":"migrating_IntegerInstantiation","enabled":true},{"patternId":"design_NonStaticInitializer","enabled":true},{"patternId":"design_MissingBreakInSwitch","enabled":true},{"patternId":"design_AvoidReassigningParameters","enabled":true},{"patternId":"basic_AvoidThreadGroup","enabled":true},{"patternId":"empty_EmptyCatchBlock","parameters":{"allowCommentedBlocks":"true"},"enabled":true},{"patternId":"codesize_ExcessiveParameterList","parameters":{"minimum":"8","violationSuppressRegex":"\"\"","violationSuppressXPath":"\"\""},"enabled":true},{"patternId":"naming_SuspiciousHashcodeMethodName","enabled":true},{"patternId":"migrating_JUnit4TestShouldUseBeforeAnnotation","enabled":true},{"patternId":"design_UncommentedEmptyMethodBody","enabled":true},{"patternId":"basic_BrokenNullCheck","enabled":true},{"patternId":"strings_ConsecutiveLiteralAppends","enabled":true},{"patternId":"strings_StringInstantiation","enabled":true},{"patternId":"design_EqualsNull","enabled":true},{"patternId":"basic_OverrideBothEqualsAndHashcode","enabled":true},{"patternId":"design_InstantiationToGetClass","enabled":true},{"patternId":"basic_BooleanInstantiation","enabled":true},{"patternId":"strings_AvoidStringBufferField","enabled":true},{"patternId":"basic_ReturnFromFinallyBlock","enabled":true},{"patternId":"empty_EmptyTryBlock","enabled":true},{"patternId":"naming_SuspiciousEqualsMethodName","enabled":true},{"patternId":"basic_ExtendsObject","enabled":true},{"patternId":"strings_UselessStringValueOf","enabled":true},{"patternId":"design_UnsynchronizedStaticDateFormatter","enabled":true},{"patternId":"design_UseCollectionIsEmpty","enabled":true},{"patternId":"controversial_AvoidFinalLocalVariable","enabled":true},{"patternId":"strictexception_AvoidThrowingNullPointerException","enabled":true},{"patternId":"design_AvoidProtectedMethodInFinalClassNotExtending","enabled":true},{"patternId":"optimizations_PrematureDeclaration","enabled":true},{"patternId":"empty_EmptySwitchStatements","enabled":true},{"patternId":"basic_MisplacedNullCheck","enabled":true},{"patternId":"optimizations_UseStringBufferForStringAppends","enabled":true},{"patternId":"strings_StringToString","enabled":true},{"patternId":"naming_MethodWithSameNameAsEnclosingClass","enabled":true},{"patternId":"migrating_ReplaceVectorWithList","enabled":true},{"patternId":"imports_UnusedImports","enabled":true},{"patternId":"unnecessary_UnnecessaryFinalModifier","enabled":true},{"patternId":"basic_AvoidMultipleUnaryOperators","enabled":true},{"patternId":"junit_SimplifyBooleanAssertion","enabled":true},{"patternId":"unnecessary_UselessParentheses","enabled":true},{"patternId":"design_IdempotentOperations","enabled":true},{"patternId":"braces_IfStmtsMustUseBraces","enabled":true},{"patternId":"strings_UseIndexOfChar","enabled":true},{"patternId":"naming_NoPackage","enabled":true},{"patternId":"finalizers_FinalizeDoesNotCallSuperFinalize","enabled":true},{"patternId":"design_UseVarargs","enabled":true},{"patternId":"unusedcode_UnusedFormalParameter","enabled":true},{"patternId":"design_ReturnEmptyArrayRatherThanNull","enabled":true},{"patternId":"junit_UseAssertNullInsteadOfAssertTrue","enabled":true},{"patternId":"design_UseUtilityClass","enabled":true},{"patternId":"design_AvoidDeeplyNestedIfStmts","enabled":true},{"patternId":"empty_EmptyStatementNotInLoop","enabled":true},{"patternId":"junit_UseAssertSameInsteadOfAssertTrue","enabled":true},{"patternId":"braces_ForLoopsMustUseBraces","enabled":true},{"patternId":"controversial_DoNotCallGarbageCollectionExplicitly","enabled":true},{"patternId":"naming_GenericsNaming","enabled":true},{"patternId":"strings_UseEqualsToCompareStrings","enabled":true},{"patternId":"optimizations_AvoidArrayLoops","enabled":true},{"patternId":"empty_EmptyStaticInitializer","enabled":true},{"patternId":"design_UncommentedEmptyConstructor","enabled":true},{"patternId":"empty_EmptyStatementBlock","enabled":true},{"patternId":"basic_CollapsibleIfStatements","enabled":true},{"patternId":"design_FinalFieldCouldBeStatic","enabled":true},{"patternId":"logging-java_MoreThanOneLogger","enabled":true},{"patternId":"codesize_ExcessiveClassLength","enabled":true},{"patternId":"design_ImmutableField","enabled":true},{"patternId":"controversial_OneDeclarationPerLine","enabled":true},{"patternId":"empty_EmptyWhileStmt","enabled":true},{"patternId":"unnecessary_UnnecessaryReturn","enabled":true},{"patternId":"strings_InefficientEmptyStringCheck","enabled":true},{"patternId":"design_UseNotifyAllInsteadOfNotify","enabled":true},{"patternId":"strictexception_DoNotThrowExceptionInFinally","enabled":true},{"patternId":"junit_UseAssertEqualsInsteadOfAssertTrue","enabled":true},{"patternId":"typeresolution_CloneMethodMustImplementCloneable","enabled":true},{"patternId":"codesize_NPathComplexity","enabled":true},{"patternId":"imports_DontImportJavaLang","enabled":true},{"patternId":"empty_EmptySynchronizedBlock","enabled":true},{"patternId":"migrating_JUnit4TestShouldUseAfterAnnotation","enabled":true},{"patternId":"design_AvoidConstantsInterface","enabled":true},{"patternId":"unnecessary_UselessOperationOnImmutable","enabled":true},{"patternId":"design_PositionLiteralsFirstInComparisons","enabled":true},{"patternId":"migrating_ByteInstantiation","enabled":true},{"patternId":"junit_JUnitSpelling","enabled":true},{"patternId":"junit_JUnitTestsShouldIncludeAssert","enabled":true},{"patternId":"finalizers_EmptyFinalizer","enabled":true},{"patternId":"design_NonCaseLabelInSwitchStatement","enabled":true},{"patternId":"android_DoNotHardCodeSDCard","enabled":true},{"patternId":"design_LogicInversion","enabled":true},{"patternId":"unusedcode_UnusedPrivateMethod","enabled":true},{"patternId":"naming_AvoidDollarSigns","enabled":true},{"patternId":"finalizers_FinalizeShouldBeProtected","enabled":true},{"patternId":"clone_ProperCloneImplementation","enabled":true},{"patternId":"basic_CheckResultSet","enabled":true},{"patternId":"controversial_AvoidPrefixingMethodParameters","enabled":true},{"patternId":"migrating_JUnit4SuitesShouldUseSuiteAnnotation","enabled":true},{"patternId":"empty_EmptyIfStmt","enabled":true},{"patternId":"basic_DontCallThreadRun","enabled":true},{"patternId":"junit_JUnitStaticSuite","enabled":true},{"patternId":"optimizations_UseArraysAsList","enabled":true},{"patternId":"design_MissingStaticMethodInNonInstantiatableClass","enabled":true},{"patternId":"unusedcode_UnusedModifier","enabled":true},{"patternId":"Style_MethodName","enabled":true},{"patternId":"Metrics_CyclomaticComplexity","enabled":true},{"patternId":"Lint_DuplicateMethods","enabled":true},{"patternId":"Style_Lambda","enabled":true},{"patternId":"Lint_UselessSetterCall","enabled":true},{"patternId":"Style_VariableName","enabled":true},{"patternId":"Lint_AmbiguousOperator","enabled":true},{"patternId":"Style_LeadingCommentSpace","enabled":true},{"patternId":"Style_CaseEquality","enabled":true},{"patternId":"Lint_StringConversionInInterpolation","enabled":true},{"patternId":"Performance_ReverseEach","enabled":true},{"patternId":"Lint_LiteralInCondition","enabled":true},{"patternId":"Performance_Sample","enabled":true},{"patternId":"Style_NonNilCheck","enabled":true},{"patternId":"Lint_RescueException","enabled":true},{"patternId":"Lint_UselessElseWithoutRescue","enabled":true},{"patternId":"Style_ConstantName","enabled":true},{"patternId":"Lint_LiteralInInterpolation","enabled":true},{"patternId":"Lint_NestedMethodDefinition","enabled":true},{"patternId":"Style_DoubleNegation","enabled":true},{"patternId":"Lint_SpaceBeforeFirstArg","enabled":true},{"patternId":"Lint_Debugger","enabled":true},{"patternId":"Style_ClassVars","enabled":true},{"patternId":"Lint_EmptyEnsure","enabled":true},{"patternId":"Style_MultilineBlockLayout","enabled":true},{"patternId":"Lint_UnusedBlockArgument","enabled":true},{"patternId":"Lint_UselessAccessModifier","enabled":true},{"patternId":"Performance_Size","enabled":true},{"patternId":"Lint_EachWithObjectArgument","enabled":true},{"patternId":"Style_Alias","enabled":true},{"patternId":"Lint_Loop","enabled":true},{"patternId":"Style_NegatedWhile","enabled":true},{"patternId":"Style_ColonMethodCall","enabled":true},{"patternId":"Lint_AmbiguousRegexpLiteral","enabled":true},{"patternId":"Lint_UnusedMethodArgument","enabled":true},{"patternId":"Style_MultilineIfThen","enabled":true},{"patternId":"Lint_EnsureReturn","enabled":true},{"patternId":"Style_NegatedIf","enabled":true},{"patternId":"Lint_Eval","enabled":true},{"patternId":"Style_NilComparison","enabled":true},{"patternId":"Style_ArrayJoin","enabled":true},{"patternId":"Lint_ConditionPosition","enabled":true},{"patternId":"Lint_UnreachableCode","enabled":true},{"patternId":"Performance_Count","enabled":true},{"patternId":"Lint_EmptyInterpolation","enabled":true},{"patternId":"Style_LambdaCall","enabled":true},{"patternId":"Lint_HandleExceptions","enabled":true},{"patternId":"Lint_ShadowingOuterLocalVariable","enabled":true},{"patternId":"Lint_EndAlignment","enabled":true},{"patternId":"Style_MultilineTernaryOperator","enabled":true},{"patternId":"Style_AutoResourceCleanup","enabled":true},{"patternId":"Lint_ElseLayout","enabled":true},{"patternId":"Style_NestedTernaryOperator","enabled":true},{"patternId":"Style_OneLineConditional","enabled":true},{"patternId":"Style_EmptyElse","enabled":true},{"patternId":"Lint_UselessComparison","enabled":true},{"patternId":"Metrics_PerceivedComplexity","enabled":true},{"patternId":"Style_InfiniteLoop","enabled":true},{"patternId":"Rails_Date","enabled":true},{"patternId":"Style_EvenOdd","enabled":true},{"patternId":"Style_IndentationConsistency","enabled":true},{"patternId":"Style_ModuleFunction","enabled":true},{"patternId":"Lint_UselessAssignment","enabled":true},{"patternId":"Style_EachWithObject","enabled":true},{"patternId":"Performance_Detect","enabled":true},{"patternId":"duplicate_key","enabled":true},{"patternId":"no_interpolation_in_single_quotes","enabled":true},{"patternId":"no_backticks","enabled":true},{"patternId":"no_unnecessary_fat_arrows","enabled":true},{"patternId":"indentation","enabled":true},{"patternId":"ensure_comprehensions","enabled":true},{"patternId":"no_stand_alone_at","enabled":true},{"patternId":"cyclomatic_complexity","enabled":true},{"patternId":"Deserialize","enabled":true},{"patternId":"SymbolDoS","enabled":true},{"patternId":"SkipBeforeFilter","enabled":true},{"patternId":"SanitizeMethods","enabled":true},{"patternId":"SelectTag","enabled":true},{"patternId":"XMLDoS","enabled":true},{"patternId":"SimpleFormat","enabled":true},{"patternId":"Evaluation","enabled":true},{"patternId":"BasicAuth","enabled":true},{"patternId":"JRubyXML","enabled":true},{"patternId":"RenderInline","enabled":true},{"patternId":"YAMLParsing","enabled":true},{"patternId":"Redirect","enabled":true},{"patternId":"UnsafeReflection","enabled":true},{"patternId":"SSLVerify","enabled":true},{"patternId":"HeaderDoS","enabled":true},{"patternId":"TranslateBug","enabled":true},{"patternId":"Execute","enabled":true},{"patternId":"JSONParsing","enabled":true},{"patternId":"LinkTo","enabled":true},{"patternId":"FileDisclosure","enabled":true},{"patternId":"SafeBufferManipulation","enabled":true},{"patternId":"ModelAttributes","enabled":true},{"patternId":"ResponseSplitting","enabled":true},{"patternId":"DigestDoS","enabled":true},{"patternId":"Send","enabled":true},{"patternId":"MailTo","enabled":true},{"patternId":"SymbolDoSCVE","enabled":true},{"patternId":"StripTags","enabled":true},{"patternId":"MassAssignment","enabled":true},{"patternId":"RegexDoS","enabled":true},{"patternId":"SelectVulnerability","enabled":true},{"patternId":"FileAccess","enabled":true},{"patternId":"ContentTag","enabled":true},{"patternId":"SessionSettings","enabled":true},{"patternId":"FilterSkipping","enabled":true},{"patternId":"CreateWith","enabled":true},{"patternId":"JSONEncoding","enabled":true},{"patternId":"SQLCVEs","enabled":true},{"patternId":"ForgerySetting","enabled":true},{"patternId":"QuoteTableName","enabled":true},{"patternId":"I18nXSS","enabled":true},{"patternId":"WithoutProtection","enabled":true},{"patternId":"CrossSiteScripting","enabled":true},{"patternId":"SingleQuotes","enabled":true},{"patternId":"NestedAttributes","enabled":true},{"patternId":"DetailedExceptions","enabled":true},{"patternId":"LinkToHref","enabled":true},{"patternId":"RenderDoS","enabled":true},{"patternId":"ModelSerialize","enabled":true},{"patternId":"SQL","enabled":true},{"patternId":"Render","enabled":true},{"patternId":"UnscopedFind","enabled":true},{"patternId":"ValidationRegex","enabled":true},{"patternId":"EscapeFunction","enabled":true},{"patternId":"Custom_Scala_FieldNamesChecker","enabled":true},{"patternId":"Custom_Scala_ObjDeserialization","enabled":true},{"patternId":"Custom_Scala_RSAPadding","enabled":true},{"patternId":"ESLint_no-extra-boolean-cast","enabled":true},{"patternId":"ESLint_no-iterator","enabled":true},{"patternId":"ESLint_no-invalid-regexp","enabled":true},{"patternId":"ESLint_no-obj-calls","enabled":true},{"patternId":"ESLint_no-sparse-arrays","enabled":true},{"patternId":"ESLint_no-unreachable","enabled":true},{"patternId":"ESLint_no-dupe-keys","enabled":true},{"patternId":"ESLint_no-multi-str","enabled":true},{"patternId":"ESLint_no-extend-native","enabled":true},{"patternId":"ESLint_guard-for-in","enabled":true},{"patternId":"ESLint_no-func-assign","enabled":true},{"patternId":"ESLint_no-extra-semi","enabled":true},{"patternId":"ESLint_camelcase","enabled":true},{"patternId":"ESLint_no-mixed-spaces-and-tabs","enabled":true},{"patternId":"ESLint_no-undef","enabled":true},{"patternId":"ESLint_semi","enabled":true},{"patternId":"ESLint_no-empty-character-class","enabled":true},{"patternId":"ESLint_complexity","enabled":true},{"patternId":"ESLint_no-dupe-class-members","enabled":true},{"patternId":"ESLint_no-debugger","enabled":true},{"patternId":"ESLint_block-scoped-var","enabled":true},{"patternId":"ESLint_no-loop-func","enabled":true},{"patternId":"ESLint_no-use-before-define","enabled":true},{"patternId":"ESLint_no-console","enabled":true},{"patternId":"ESLint_require-yield","enabled":true},{"patternId":"ESLint_no-redeclare","enabled":true},{"patternId":"ESLint_no-undefined","enabled":true},{"patternId":"ESLint_use-isnan","enabled":true},{"patternId":"ESLint_no-control-regex","enabled":true},{"patternId":"ESLint_no-const-assign","enabled":true},{"patternId":"ESLint_no-new","enabled":true},{"patternId":"ESLint_new-cap","enabled":true},{"patternId":"ESLint_no-irregular-whitespace","enabled":true},{"patternId":"ESLint_object-shorthand","enabled":true},{"patternId":"ESLint_no-ex-assign","enabled":true},{"patternId":"ESLint_wrap-iife","enabled":true},{"patternId":"ESLint_arrow-parens","enabled":true},{"patternId":"ESLint_no-constant-condition","enabled":true},{"patternId":"ESLint_no-octal","enabled":true},{"patternId":"ESLint_no-dupe-args","enabled":true},{"patternId":"ESLint_quotes","enabled":true},{"patternId":"ESLint_no-fallthrough","enabled":true},{"patternId":"ESLint_no-delete-var","enabled":true},{"patternId":"ESLint_no-caller","enabled":true},{"patternId":"ESLint_no-cond-assign","enabled":true},{"patternId":"ESLint_no-this-before-super","enabled":true},{"patternId":"ESLint_no-negated-in-lhs","enabled":true},{"patternId":"ESLint_no-inner-declarations","enabled":true},{"patternId":"ESLint_eqeqeq","enabled":true},{"patternId":"ESLint_curly","enabled":true},{"patternId":"ESLint_arrow-spacing","enabled":true},{"patternId":"ESLint_no-empty","enabled":true},{"patternId":"ESLint_no-unused-vars","enabled":true},{"patternId":"ESLint_generator-star-spacing","enabled":true},{"patternId":"ESLint_no-duplicate-case","enabled":true},{"patternId":"ESLint_valid-typeof","enabled":true},{"patternId":"ESLint_no-regex-spaces","enabled":true},{"patternId":"ESLint_no-class-assign","enabled":true},{"patternId":"PyLint_W0221","enabled":true},{"patternId":"PyLint_E0117","enabled":true},{"patternId":"PyLint_E0001","enabled":true},{"patternId":"PyLint_E0241","enabled":true},{"patternId":"PyLint_W0404","enabled":true},{"patternId":"PyLint_E0704","enabled":true},{"patternId":"PyLint_E0703","enabled":true},{"patternId":"PyLint_E0302","enabled":true},{"patternId":"PyLint_W1301","enabled":true},{"patternId":"PyLint_R0201","enabled":true},{"patternId":"PyLint_E0113","enabled":true},{"patternId":"PyLint_W0410","enabled":true},{"patternId":"PyLint_C0123","enabled":true},{"patternId":"PyLint_E0115","enabled":true},{"patternId":"PyLint_E0114","enabled":true},{"patternId":"PyLint_E1126","enabled":true},{"patternId":"PyLint_W0702","enabled":true},{"patternId":"PyLint_W1303","enabled":true},{"patternId":"PyLint_W0622","enabled":true},{"patternId":"PyLint_W0222","enabled":true},{"patternId":"PyLint_W0233","enabled":true},{"patternId":"PyLint_W1305","enabled":true},{"patternId":"PyLint_E1127","enabled":true},{"patternId":"PyLint_E0112","enabled":true},{"patternId":"PyLint_W0611","enabled":true},{"patternId":"PyLint_W0601","enabled":true},{"patternId":"PyLint_W1300","enabled":true},{"patternId":"PyLint_W0124","enabled":true},{"patternId":"PyLint_R0203","enabled":true},{"patternId":"PyLint_E0236","enabled":true},{"patternId":"PyLint_W0612","enabled":true},{"patternId":"PyLint_W0604","enabled":true},{"patternId":"PyLint_W0705","enabled":true},{"patternId":"PyLint_E0238","enabled":true},{"patternId":"PyLint_W0602","enabled":true},{"patternId":"PyLint_R0102","enabled":true},{"patternId":"PyLint_R0202","enabled":true},{"patternId":"PyLint_E0240","enabled":true},{"patternId":"PyLint_W0623","enabled":true},{"patternId":"PyLint_W0711","enabled":true},{"patternId":"PyLint_E0116","enabled":true},{"patternId":"PyLint_E0239","enabled":true},{"patternId":"PyLint_E1132","enabled":true},{"patternId":"PyLint_W1307","enabled":true},{"patternId":"PyLint_C0200","enabled":true},{"patternId":"PyLint_E0301","enabled":true},{"patternId":"PyLint_W1306","enabled":true},{"patternId":"PyLint_W1302","enabled":true},{"patternId":"PyLint_E0110","enabled":true},{"patternId":"PyLint_E1125","enabled":true}]} \ No newline at end of file diff --git a/gcloud-java-bigquery/README.md b/gcloud-java-bigquery/README.md index df1058dec024..81b5db71bcac 100644 --- a/gcloud-java-bigquery/README.md +++ b/gcloud-java-bigquery/README.md @@ -7,6 +7,7 @@ Java idiomatic client for [Google Cloud BigQuery] (https://cloud.google.com/bigq [![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) [![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-bigquery.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-bigquery.svg) [![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) - [Homepage] (https://googlecloudplatform.github.io/gcloud-java/) - [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/bigquery/package-summary.html) @@ -21,22 +22,22 @@ If you are using Maven, add this to your pom.xml file com.google.gcloud gcloud-java-bigquery - 0.1.3 + 0.1.5 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.gcloud:gcloud-java-bigquery:0.1.3' +compile 'com.google.gcloud:gcloud-java-bigquery:0.1.5' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.gcloud" % "gcloud-java-bigquery" % "0.1.3" +libraryDependencies += "com.google.gcloud" % "gcloud-java-bigquery" % "0.1.5" ``` Example Application ------------------- - [`BigQueryExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/BigQueryExample.java) - A simple command line interface providing some of Cloud BigQuery's functionality. -Read more about using this application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/BigQueryExample.html). +Read more about using this application on the [`BigQueryExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/bigquery/BigQueryExample.html). Authentication -------------- @@ -184,7 +185,7 @@ Then add the following code to run the query and wait for the result: QueryRequest queryRequest = QueryRequest.builder("SELECT * FROM my_dataset_id.my_table_id") .maxWaitTime(60000L) - .maxResults(1000L) + .pageSize(1000L) .build(); // Request query to be executed and wait for results QueryResponse queryResponse = bigquery.query(queryRequest); diff --git a/gcloud-java-bigquery/pom.xml b/gcloud-java-bigquery/pom.xml index 41a51722b7c6..5c79f150c722 100644 --- a/gcloud-java-bigquery/pom.xml +++ b/gcloud-java-bigquery/pom.xml @@ -10,7 +10,7 @@ com.google.gcloud gcloud-java-pom - 0.1.4-SNAPSHOT + 0.1.6-SNAPSHOT gcloud-java-bigquery @@ -30,15 +30,22 @@ com.google.apis google-api-services-bigquery - v2-rev254-1.21.0 + v2-rev270-1.21.0 compile - - com.google.guava - guava-jdk5 - + + com.google.guava + guava-jdk5 + + + ${project.groupId} + gcloud-java-core + ${project.version} + test-jar + test + junit junit @@ -48,7 +55,7 @@ org.easymock easymock - 3.3 + 3.4 test diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQuery.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQuery.java index 4b5d3ef0c81a..14e324a43370 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQuery.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQuery.java @@ -25,7 +25,7 @@ import com.google.common.collect.Sets; import com.google.gcloud.Page; import com.google.gcloud.Service; -import com.google.gcloud.spi.BigQueryRpc; +import com.google.gcloud.bigquery.spi.BigQueryRpc; import java.util.List; import java.util.Set; @@ -171,16 +171,16 @@ private DatasetListOption(BigQueryRpc.Option option, Object value) { } /** - * Returns an option to specify the maximum number of datasets to be returned. + * Returns an option to specify the maximum number of datasets returned per page. */ - public static DatasetListOption maxResults(long maxResults) { - return new DatasetListOption(BigQueryRpc.Option.MAX_RESULTS, maxResults); + public static DatasetListOption pageSize(long pageSize) { + return new DatasetListOption(BigQueryRpc.Option.MAX_RESULTS, pageSize); } /** * Returns an option to specify the page token from which to start listing datasets. */ - public static DatasetListOption startPageToken(String pageToken) { + public static DatasetListOption pageToken(String pageToken) { return new DatasetListOption(BigQueryRpc.Option.PAGE_TOKEN, pageToken); } @@ -246,17 +246,17 @@ private TableListOption(BigQueryRpc.Option option, Object value) { } /** - * Returns an option to specify the maximum number of tables to be returned. + * Returns an option to specify the maximum number of tables returned per page. */ - public static TableListOption maxResults(long maxResults) { - checkArgument(maxResults >= 0); - return new TableListOption(BigQueryRpc.Option.MAX_RESULTS, maxResults); + public static TableListOption pageSize(long pageSize) { + checkArgument(pageSize >= 0); + return new TableListOption(BigQueryRpc.Option.MAX_RESULTS, pageSize); } /** * Returns an option to specify the page token from which to start listing tables. */ - public static TableListOption startPageToken(String pageToken) { + public static TableListOption pageToken(String pageToken) { return new TableListOption(BigQueryRpc.Option.PAGE_TOKEN, pageToken); } } @@ -295,17 +295,17 @@ private TableDataListOption(BigQueryRpc.Option option, Object value) { } /** - * Returns an option to specify the maximum number of rows to be returned. + * Returns an option to specify the maximum number of rows returned per page. */ - public static TableDataListOption maxResults(long maxResults) { - checkArgument(maxResults >= 0); - return new TableDataListOption(BigQueryRpc.Option.MAX_RESULTS, maxResults); + public static TableDataListOption pageSize(long pageSize) { + checkArgument(pageSize >= 0); + return new TableDataListOption(BigQueryRpc.Option.MAX_RESULTS, pageSize); } /** * Returns an option to specify the page token from which to start listing table data. */ - public static TableDataListOption startPageToken(String pageToken) { + public static TableDataListOption pageToken(String pageToken) { return new TableDataListOption(BigQueryRpc.Option.PAGE_TOKEN, pageToken); } @@ -352,17 +352,17 @@ public String apply(JobStatus.State state) { } /** - * Returns an option to specify the maximum number of jobs to be returned. + * Returns an option to specify the maximum number of jobs returned per page. */ - public static JobListOption maxResults(long maxResults) { - checkArgument(maxResults >= 0); - return new JobListOption(BigQueryRpc.Option.MAX_RESULTS, maxResults); + public static JobListOption pageSize(long pageSize) { + checkArgument(pageSize >= 0); + return new JobListOption(BigQueryRpc.Option.MAX_RESULTS, pageSize); } /** * Returns an option to specify the page token from which to start listing jobs. */ - public static JobListOption startPageToken(String pageToken) { + public static JobListOption pageToken(String pageToken) { return new JobListOption(BigQueryRpc.Option.PAGE_TOKEN, pageToken); } @@ -418,17 +418,17 @@ private QueryResultsOption(BigQueryRpc.Option option, Object value) { } /** - * Returns an option to specify the maximum number of rows to be returned. + * Returns an option to specify the maximum number of rows returned per page. */ - public static QueryResultsOption maxResults(long maxResults) { - checkArgument(maxResults >= 0); - return new QueryResultsOption(BigQueryRpc.Option.MAX_RESULTS, maxResults); + public static QueryResultsOption pageSize(long pageSize) { + checkArgument(pageSize >= 0); + return new QueryResultsOption(BigQueryRpc.Option.MAX_RESULTS, pageSize); } /** * Returns an option to specify the page token from which to start getting query results. */ - public static QueryResultsOption startPageToken(String pageToken) { + public static QueryResultsOption pageToken(String pageToken) { return new QueryResultsOption(BigQueryRpc.Option.PAGE_TOKEN, pageToken); } @@ -457,35 +457,35 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * * @throws BigQueryException upon failure */ - Dataset create(DatasetInfo dataset, DatasetOption... options) throws BigQueryException; + Dataset create(DatasetInfo dataset, DatasetOption... options); /** * Creates a new table. * * @throws BigQueryException upon failure */ - Table create(TableInfo table, TableOption... options) throws BigQueryException; + Table create(TableInfo table, TableOption... options); /** * Creates a new job. * * @throws BigQueryException upon failure */ - Job create(JobInfo job, JobOption... options) throws BigQueryException; + Job create(JobInfo job, JobOption... options); /** * Returns the requested dataset or {@code null} if not found. * * @throws BigQueryException upon failure */ - Dataset getDataset(String datasetId, DatasetOption... options) throws BigQueryException; + Dataset getDataset(String datasetId, DatasetOption... options); /** * Returns the requested dataset or {@code null} if not found. * * @throws BigQueryException upon failure */ - Dataset getDataset(DatasetId datasetId, DatasetOption... options) throws BigQueryException; + Dataset getDataset(DatasetId datasetId, DatasetOption... options); /** * Lists the project's datasets. This method returns partial information on each dataset @@ -495,7 +495,7 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * * @throws BigQueryException upon failure */ - Page listDatasets(DatasetListOption... options) throws BigQueryException; + Page listDatasets(DatasetListOption... options); /** * Deletes the requested dataset. @@ -503,7 +503,7 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * @return {@code true} if dataset was deleted, {@code false} if it was not found * @throws BigQueryException upon failure */ - boolean delete(String datasetId, DatasetDeleteOption... options) throws BigQueryException; + boolean delete(String datasetId, DatasetDeleteOption... options); /** * Deletes the requested dataset. @@ -511,7 +511,7 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * @return {@code true} if dataset was deleted, {@code false} if it was not found * @throws BigQueryException upon failure */ - boolean delete(DatasetId datasetId, DatasetDeleteOption... options) throws BigQueryException; + boolean delete(DatasetId datasetId, DatasetDeleteOption... options); /** * Deletes the requested table. @@ -519,7 +519,7 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * @return {@code true} if table was deleted, {@code false} if it was not found * @throws BigQueryException upon failure */ - boolean delete(String datasetId, String tableId) throws BigQueryException; + boolean delete(String datasetId, String tableId); /** * Deletes the requested table. @@ -527,35 +527,35 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * @return {@code true} if table was deleted, {@code false} if it was not found * @throws BigQueryException upon failure */ - boolean delete(TableId tableId) throws BigQueryException; + boolean delete(TableId tableId); /** * Updates dataset information. * * @throws BigQueryException upon failure */ - Dataset update(DatasetInfo dataset, DatasetOption... options) throws BigQueryException; + Dataset update(DatasetInfo dataset, DatasetOption... options); /** * Updates table information. * * @throws BigQueryException upon failure */ - Table update(TableInfo table, TableOption... options) throws BigQueryException; + Table update(TableInfo table, TableOption... options); /** * Returns the requested table or {@code null} if not found. * * @throws BigQueryException upon failure */ - Table getTable(String datasetId, String tableId, TableOption... options) throws BigQueryException; + Table getTable(String datasetId, String tableId, TableOption... options); /** * Returns the requested table or {@code null} if not found. * * @throws BigQueryException upon failure */ - Table getTable(TableId tableId, TableOption... options) throws BigQueryException; + Table getTable(TableId tableId, TableOption... options); /** * Lists the tables in the dataset. This method returns partial information on each table @@ -566,7 +566,7 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * * @throws BigQueryException upon failure */ - Page listTables(String datasetId, TableListOption... options) throws BigQueryException; + Page
listTables(String datasetId, TableListOption... options); /** * Lists the tables in the dataset. This method returns partial information on each table @@ -577,14 +577,14 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * * @throws BigQueryException upon failure */ - Page
listTables(DatasetId datasetId, TableListOption... options) throws BigQueryException; + Page
listTables(DatasetId datasetId, TableListOption... options); /** * Sends an insert all request. * * @throws BigQueryException upon failure */ - InsertAllResponse insertAll(InsertAllRequest request) throws BigQueryException; + InsertAllResponse insertAll(InsertAllRequest request); /** * Lists the table's rows. @@ -592,36 +592,35 @@ public static QueryResultsOption maxWaitTime(long maxWaitTime) { * @throws BigQueryException upon failure */ Page> listTableData(String datasetId, String tableId, - TableDataListOption... options) throws BigQueryException; + TableDataListOption... options); /** * Lists the table's rows. * * @throws BigQueryException upon failure */ - Page> listTableData(TableId tableId, TableDataListOption... options) - throws BigQueryException; + Page> listTableData(TableId tableId, TableDataListOption... options); /** * Returns the requested job or {@code null} if not found. * * @throws BigQueryException upon failure */ - Job getJob(String jobId, JobOption... options) throws BigQueryException; + Job getJob(String jobId, JobOption... options); /** * Returns the requested job or {@code null} if not found. * * @throws BigQueryException upon failure */ - Job getJob(JobId jobId, JobOption... options) throws BigQueryException; + Job getJob(JobId jobId, JobOption... options); /** * Lists the jobs. * * @throws BigQueryException upon failure */ - Page listJobs(JobListOption... options) throws BigQueryException; + Page listJobs(JobListOption... options); /** * Sends a job cancel request. This call will return immediately. The job status can then be @@ -632,7 +631,7 @@ Page> listTableData(TableId tableId, TableDataListOption... opt * found * @throws BigQueryException upon failure */ - boolean cancel(String jobId) throws BigQueryException; + boolean cancel(String jobId); /** * Sends a job cancel request. This call will return immediately. The job status can then be @@ -643,21 +642,21 @@ Page> listTableData(TableId tableId, TableDataListOption... opt * found * @throws BigQueryException upon failure */ - boolean cancel(JobId tableId) throws BigQueryException; + boolean cancel(JobId tableId); /** * Runs the query associated with the request. * * @throws BigQueryException upon failure */ - QueryResponse query(QueryRequest request) throws BigQueryException; + QueryResponse query(QueryRequest request); /** * Returns results of the query associated with the provided job. * * @throws BigQueryException upon failure */ - QueryResponse getQueryResults(JobId job, QueryResultsOption... options) throws BigQueryException; + QueryResponse getQueryResults(JobId job, QueryResultsOption... options); /** * Returns a channel to write data to be inserted into a BigQuery table. Data format and other diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryException.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryException.java index a157afd25db2..e78734a2899e 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryException.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryException.java @@ -22,6 +22,7 @@ import com.google.gcloud.RetryHelper.RetryInterruptedException; import java.io.IOException; +import java.util.Objects; import java.util.Set; /** @@ -73,6 +74,23 @@ protected Set retryableErrors() { return RETRYABLE_ERRORS; } + @Override + public boolean equals(Object obj) { + if (obj == this) { + return true; + } + if (!(obj instanceof BigQueryException)) { + return false; + } + BigQueryException other = (BigQueryException) obj; + return super.equals(other) && Objects.equals(error, other.error); + } + + @Override + public int hashCode() { + return Objects.hash(super.hashCode(), error); + } + /** * Translate RetryHelperException to the BigQueryException that caused the error. This method will * always throw an exception. diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryImpl.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryImpl.java index 1158dd86c83d..27f4af5d5007 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryImpl.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryImpl.java @@ -35,7 +35,7 @@ import com.google.gcloud.PageImpl.NextPageFetcher; import com.google.gcloud.RetryHelper; import com.google.gcloud.bigquery.InsertAllRequest.RowToInsert; -import com.google.gcloud.spi.BigQueryRpc; +import com.google.gcloud.bigquery.spi.BigQueryRpc; import java.util.List; import java.util.Map; @@ -153,7 +153,7 @@ public QueryResult nextPage() { } @Override - public Dataset create(DatasetInfo dataset, DatasetOption... options) throws BigQueryException { + public Dataset create(DatasetInfo dataset, DatasetOption... options) { final com.google.api.services.bigquery.model.Dataset datasetPb = dataset.setProjectId(options().projectId()).toPb(); final Map optionsMap = optionMap(options); @@ -171,7 +171,7 @@ public com.google.api.services.bigquery.model.Dataset call() { } @Override - public Table create(TableInfo table, TableOption... options) throws BigQueryException { + public Table create(TableInfo table, TableOption... options) { final com.google.api.services.bigquery.model.Table tablePb = table.setProjectId(options().projectId()).toPb(); final Map optionsMap = optionMap(options); @@ -189,7 +189,7 @@ public com.google.api.services.bigquery.model.Table call() { } @Override - public Job create(JobInfo job, JobOption... options) throws BigQueryException { + public Job create(JobInfo job, JobOption... options) { final com.google.api.services.bigquery.model.Job jobPb = job.setProjectId(options().projectId()).toPb(); final Map optionsMap = optionMap(options); @@ -207,13 +207,12 @@ public com.google.api.services.bigquery.model.Job call() { } @Override - public Dataset getDataset(String datasetId, DatasetOption... options) throws BigQueryException { + public Dataset getDataset(String datasetId, DatasetOption... options) { return getDataset(DatasetId.of(datasetId), options); } @Override - public Dataset getDataset(final DatasetId datasetId, DatasetOption... options) - throws BigQueryException { + public Dataset getDataset(final DatasetId datasetId, DatasetOption... options) { final Map optionsMap = optionMap(options); try { com.google.api.services.bigquery.model.Dataset answer = @@ -230,7 +229,7 @@ public com.google.api.services.bigquery.model.Dataset call() { } @Override - public Page listDatasets(DatasetListOption... options) throws BigQueryException { + public Page listDatasets(DatasetListOption... options) { return listDatasets(options(), optionMap(options)); } @@ -261,13 +260,12 @@ public Dataset apply(com.google.api.services.bigquery.model.Dataset dataset) { } @Override - public boolean delete(String datasetId, DatasetDeleteOption... options) throws BigQueryException { + public boolean delete(String datasetId, DatasetDeleteOption... options) { return delete(DatasetId.of(datasetId), options); } @Override - public boolean delete(final DatasetId datasetId, DatasetDeleteOption... options) - throws BigQueryException { + public boolean delete(final DatasetId datasetId, DatasetDeleteOption... options) { final Map optionsMap = optionMap(options); try { return runWithRetries(new Callable() { @@ -282,12 +280,12 @@ public Boolean call() { } @Override - public boolean delete(String datasetId, String tableId) throws BigQueryException { + public boolean delete(String datasetId, String tableId) { return delete(TableId.of(datasetId, tableId)); } @Override - public boolean delete(final TableId tableId) throws BigQueryException { + public boolean delete(final TableId tableId) { try { return runWithRetries(new Callable() { @Override @@ -301,7 +299,7 @@ public Boolean call() { } @Override - public Dataset update(DatasetInfo dataset, DatasetOption... options) throws BigQueryException { + public Dataset update(DatasetInfo dataset, DatasetOption... options) { final com.google.api.services.bigquery.model.Dataset datasetPb = dataset.setProjectId(options().projectId()).toPb(); final Map optionsMap = optionMap(options); @@ -319,7 +317,7 @@ public com.google.api.services.bigquery.model.Dataset call() { } @Override - public Table update(TableInfo table, TableOption... options) throws BigQueryException { + public Table update(TableInfo table, TableOption... options) { final com.google.api.services.bigquery.model.Table tablePb = table.setProjectId(options().projectId()).toPb(); final Map optionsMap = optionMap(options); @@ -337,13 +335,12 @@ public com.google.api.services.bigquery.model.Table call() { } @Override - public Table getTable(final String datasetId, final String tableId, TableOption... options) - throws BigQueryException { + public Table getTable(final String datasetId, final String tableId, TableOption... options) { return getTable(TableId.of(datasetId, tableId), options); } @Override - public Table getTable(final TableId tableId, TableOption... options) throws BigQueryException { + public Table getTable(final TableId tableId, TableOption... options) { final Map optionsMap = optionMap(options); try { com.google.api.services.bigquery.model.Table answer = @@ -360,14 +357,12 @@ public com.google.api.services.bigquery.model.Table call() { } @Override - public Page
listTables(String datasetId, TableListOption... options) - throws BigQueryException { + public Page
listTables(String datasetId, TableListOption... options) { return listTables(datasetId, options(), optionMap(options)); } @Override - public Page
listTables(DatasetId datasetId, TableListOption... options) - throws BigQueryException { + public Page
listTables(DatasetId datasetId, TableListOption... options) { return listTables(datasetId.dataset(), options(), optionMap(options)); } @@ -399,7 +394,7 @@ public Table apply(com.google.api.services.bigquery.model.Table table) { } @Override - public InsertAllResponse insertAll(InsertAllRequest request) throws BigQueryException { + public InsertAllResponse insertAll(InsertAllRequest request) { final TableId tableId = request.table(); final TableDataInsertAllRequest requestPb = new TableDataInsertAllRequest(); requestPb.setIgnoreUnknownValues(request.ignoreUnknownValues()); @@ -418,13 +413,12 @@ public Rows apply(RowToInsert rowToInsert) { @Override public Page> listTableData(String datasetId, String tableId, - TableDataListOption... options) throws BigQueryException { + TableDataListOption... options) { return listTableData(TableId.of(datasetId, tableId), options(), optionMap(options)); } @Override - public Page> listTableData(TableId tableId, TableDataListOption... options) - throws BigQueryException { + public Page> listTableData(TableId tableId, TableDataListOption... options) { return listTableData(tableId, options(), optionMap(options)); } @@ -459,12 +453,12 @@ public List apply(TableRow rowPb) { } @Override - public Job getJob(String jobId, JobOption... options) throws BigQueryException { + public Job getJob(String jobId, JobOption... options) { return getJob(JobId.of(jobId), options); } @Override - public Job getJob(final JobId jobId, JobOption... options) throws BigQueryException { + public Job getJob(final JobId jobId, JobOption... options) { final Map optionsMap = optionMap(options); try { com.google.api.services.bigquery.model.Job answer = @@ -481,7 +475,7 @@ public com.google.api.services.bigquery.model.Job call() { } @Override - public Page listJobs(JobListOption... options) throws BigQueryException { + public Page listJobs(JobListOption... options) { return listJobs(options(), optionMap(options)); } @@ -508,12 +502,12 @@ public Job apply(com.google.api.services.bigquery.model.Job job) { } @Override - public boolean cancel(String jobId) throws BigQueryException { + public boolean cancel(String jobId) { return cancel(JobId.of(jobId)); } @Override - public boolean cancel(final JobId jobId) throws BigQueryException { + public boolean cancel(final JobId jobId) { try { return runWithRetries(new Callable() { @Override @@ -527,7 +521,7 @@ public Boolean call() { } @Override - public QueryResponse query(final QueryRequest request) throws BigQueryException { + public QueryResponse query(final QueryRequest request) { try { com.google.api.services.bigquery.model.QueryResponse results = runWithRetries(new Callable() { @@ -566,8 +560,7 @@ public com.google.api.services.bigquery.model.QueryResponse call() { } @Override - public QueryResponse getQueryResults(JobId job, QueryResultsOption... options) - throws BigQueryException { + public QueryResponse getQueryResults(JobId job, QueryResultsOption... options) { Map optionsMap = optionMap(options); return getQueryResults(job, options(), optionsMap); } diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryOptions.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryOptions.java index 71d43cfbe565..d48cf646f349 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryOptions.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQueryOptions.java @@ -18,9 +18,9 @@ import com.google.common.collect.ImmutableSet; import com.google.gcloud.ServiceOptions; -import com.google.gcloud.spi.BigQueryRpc; -import com.google.gcloud.spi.BigQueryRpcFactory; -import com.google.gcloud.spi.DefaultBigQueryRpc; +import com.google.gcloud.bigquery.spi.BigQueryRpc; +import com.google.gcloud.bigquery.spi.BigQueryRpcFactory; +import com.google.gcloud.bigquery.spi.DefaultBigQueryRpc; import java.util.Set; diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/FieldValue.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/FieldValue.java index 24c4b28b7613..8b27c70db782 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/FieldValue.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/FieldValue.java @@ -20,9 +20,9 @@ import static com.google.common.base.Preconditions.checkState; import com.google.api.client.util.Data; -import com.google.api.client.util.Lists; import com.google.common.base.Function; import com.google.common.base.MoreObjects; +import com.google.common.collect.Lists; import java.io.Serializable; import java.util.List; diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/InsertAllRequest.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/InsertAllRequest.java index 6f39f20e498d..f0d61583f83f 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/InsertAllRequest.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/InsertAllRequest.java @@ -52,15 +52,15 @@ public class InsertAllRequest implements Serializable { * id used by BigQuery to detect duplicate insertion requests on a best-effort basis. * *

Example usage of creating a row to insert: - *

    {@code
-   *   List repeatedFieldValue = Arrays.asList(1L, 2L);
-   *   Map recordContent = new HashMap();
-   *   recordContent.put("subfieldName1", "value");
-   *   recordContent.put("subfieldName2", repeatedFieldValue);
-   *   Map rowContent = new HashMap();
-   *   rowContent.put("fieldName1", true);
-   *   rowContent.put("fieldName2", recordContent);
-   *   RowToInsert row = new RowToInsert("rowId", rowContent);
+   * 
 {@code
+   * List repeatedFieldValue = Arrays.asList(1L, 2L);
+   * Map recordContent = new HashMap();
+   * recordContent.put("subfieldName1", "value");
+   * recordContent.put("subfieldName2", repeatedFieldValue);
+   * Map rowContent = new HashMap();
+   * rowContent.put("fieldName1", true);
+   * rowContent.put("fieldName2", recordContent);
+   * RowToInsert row = new RowToInsert("rowId", rowContent);
    * }
* * @see @@ -177,16 +177,16 @@ public Builder addRow(RowToInsert rowToInsert) { * Adds a row to be inserted with associated id. * *

Example usage of adding a row with associated id: - *

    {@code
-     *   InsertAllRequest.Builder builder = InsertAllRequest.builder(tableId);
-     *   List repeatedFieldValue = Arrays.asList(1L, 2L);
-     *   Map recordContent = new HashMap();
-     *   recordContent.put("subfieldName1", "value");
-     *   recordContent.put("subfieldName2", repeatedFieldValue);
-     *   Map rowContent = new HashMap();
-     *   rowContent.put("fieldName1", true);
-     *   rowContent.put("fieldName2", recordContent);
-     *   builder.addRow("rowId", rowContent);
+     * 
 {@code
+     * InsertAllRequest.Builder builder = InsertAllRequest.builder(tableId);
+     * List repeatedFieldValue = Arrays.asList(1L, 2L);
+     * Map recordContent = new HashMap();
+     * recordContent.put("subfieldName1", "value");
+     * recordContent.put("subfieldName2", repeatedFieldValue);
+     * Map rowContent = new HashMap();
+     * rowContent.put("fieldName1", true);
+     * rowContent.put("fieldName2", recordContent);
+     * builder.addRow("rowId", rowContent);
      * }
*/ public Builder addRow(String id, Map content) { @@ -198,16 +198,16 @@ public Builder addRow(String id, Map content) { * Adds a row to be inserted without an associated id. * *

Example usage of adding a row without an associated id: - *

    {@code
-     *   InsertAllRequest.Builder builder = InsertAllRequest.builder(tableId);
-     *   List repeatedFieldValue = Arrays.asList(1L, 2L);
-     *   Map recordContent = new HashMap();
-     *   recordContent.put("subfieldName1", "value");
-     *   recordContent.put("subfieldName2", repeatedFieldValue);
-     *   Map rowContent = new HashMap();
-     *   rowContent.put("fieldName1", true);
-     *   rowContent.put("fieldName2", recordContent);
-     *   builder.addRow(rowContent);
+     * 
 {@code
+     * InsertAllRequest.Builder builder = InsertAllRequest.builder(tableId);
+     * List repeatedFieldValue = Arrays.asList(1L, 2L);
+     * Map recordContent = new HashMap();
+     * recordContent.put("subfieldName1", "value");
+     * recordContent.put("subfieldName2", repeatedFieldValue);
+     * Map rowContent = new HashMap();
+     * rowContent.put("fieldName1", true);
+     * rowContent.put("fieldName2", recordContent);
+     * builder.addRow(rowContent);
      * }
*/ public Builder addRow(Map content) { diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/Option.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/Option.java index d88820fe5a29..3fdc27ecab99 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/Option.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/Option.java @@ -19,7 +19,7 @@ import static com.google.common.base.Preconditions.checkNotNull; import com.google.common.base.MoreObjects; -import com.google.gcloud.spi.BigQueryRpc; +import com.google.gcloud.bigquery.spi.BigQueryRpc; import java.io.Serializable; import java.util.Objects; diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/QueryRequest.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/QueryRequest.java index 0bcfb3d4a9ae..b3522a2a6ba3 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/QueryRequest.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/QueryRequest.java @@ -35,26 +35,26 @@ * {@link QueryResponse#jobCompleted()} returns {@code true}. * *

Example usage of a query request: - *

    {@code
- *    // Substitute "field", "table" and "dataset" with real field, table and dataset identifiers
- *    QueryRequest request = QueryRequest.builder("SELECT field FROM table")
- *      .defaultDataset(DatasetId.of("dataset"))
- *      .maxWaitTime(60000L)
- *      .maxResults(1000L)
- *      .build();
- *    QueryResponse response = bigquery.query(request);
- *    while (!response.jobCompleted()) {
- *      Thread.sleep(1000);
- *      response = bigquery.getQueryResults(response.jobId());
- *    }
- *    List executionErrors = response.executionErrors();
- *    // look for errors in executionErrors
- *    QueryResult result = response.result();
- *    Iterator> rowIterator = result.iterateAll();
- *    while(rowIterator.hasNext()) {
- *      List row = rowIterator.next();
- *      // do something with row
- *    }
+ * 
 {@code
+ * // Substitute "field", "table" and "dataset" with real field, table and dataset identifiers
+ * QueryRequest request = QueryRequest.builder("SELECT field FROM table")
+ *     .defaultDataset(DatasetId.of("dataset"))
+ *     .maxWaitTime(60000L)
+ *     .pageSize(1000L)
+ *     .build();
+ * QueryResponse response = bigquery.query(request);
+ * while (!response.jobCompleted()) {
+ *   Thread.sleep(1000);
+ *   response = bigquery.getQueryResults(response.jobId());
+ * }
+ * List executionErrors = response.executionErrors();
+ * // look for errors in executionErrors
+ * QueryResult result = response.result();
+ * Iterator> rowIterator = result.iterateAll();
+ * while(rowIterator.hasNext()) {
+ *   List row = rowIterator.next();
+ *   // do something with row
+ * }
  * }
* * @see
Query @@ -65,7 +65,7 @@ public class QueryRequest implements Serializable { private static final long serialVersionUID = -8727328332415880852L; private final String query; - private final Long maxResults; + private final Long pageSize; private final DatasetId defaultDataset; private final Long maxWaitTime; private final Boolean dryRun; @@ -74,7 +74,7 @@ public class QueryRequest implements Serializable { public static final class Builder { private String query; - private Long maxResults; + private Long pageSize; private DatasetId defaultDataset; private Long maxWaitTime; private Boolean dryRun; @@ -96,8 +96,8 @@ public Builder query(String query) { * query result set is large. In addition to this limit, responses are also limited to 10 MB. * By default, there is no maximum row count, and only the byte limit applies. */ - public Builder maxResults(Long maxResults) { - this.maxResults = maxResults; + public Builder pageSize(Long pageSize) { + this.pageSize = pageSize; return this; } @@ -157,7 +157,7 @@ public QueryRequest build() { private QueryRequest(Builder builder) { query = builder.query; - maxResults = builder.maxResults; + pageSize = builder.pageSize; defaultDataset = builder.defaultDataset; maxWaitTime = builder.maxWaitTime; dryRun = builder.dryRun; @@ -174,8 +174,8 @@ public String query() { /** * Returns the maximum number of rows of data to return per page of results. */ - public Long maxResults() { - return maxResults; + public Long pageSize() { + return pageSize; } /** @@ -224,7 +224,7 @@ public Boolean useQueryCache() { public Builder toBuilder() { return new Builder() .query(query) - .maxResults(maxResults) + .pageSize(pageSize) .defaultDataset(defaultDataset) .maxWaitTime(maxWaitTime) .dryRun(dryRun) @@ -235,7 +235,7 @@ public Builder toBuilder() { public String toString() { return MoreObjects.toStringHelper(this) .add("query", query) - .add("maxResults", maxResults) + .add("pageSize", pageSize) .add("defaultDataset", defaultDataset) .add("maxWaitTime", maxWaitTime) .add("dryRun", dryRun) @@ -245,7 +245,7 @@ public String toString() { @Override public int hashCode() { - return Objects.hash(query, maxResults, defaultDataset, maxWaitTime, dryRun, useQueryCache); + return Objects.hash(query, pageSize, defaultDataset, maxWaitTime, dryRun, useQueryCache); } @Override @@ -264,8 +264,8 @@ QueryRequest setProjectId(String projectId) { com.google.api.services.bigquery.model.QueryRequest toPb() { com.google.api.services.bigquery.model.QueryRequest queryRequestPb = new com.google.api.services.bigquery.model.QueryRequest().setQuery(query); - if (maxResults != null) { - queryRequestPb.setMaxResults(maxResults); + if (pageSize != null) { + queryRequestPb.setMaxResults(pageSize); } if (defaultDataset != null) { queryRequestPb.setDefaultDataset(defaultDataset.toPb()); @@ -299,7 +299,7 @@ public static QueryRequest of(String query) { static QueryRequest fromPb(com.google.api.services.bigquery.model.QueryRequest queryRequestPb) { Builder builder = builder(queryRequestPb.getQuery()); if (queryRequestPb.getMaxResults() != null) { - builder.maxResults(queryRequestPb.getMaxResults()); + builder.pageSize(queryRequestPb.getMaxResults()); } if (queryRequestPb.getDefaultDataset() != null) { builder.defaultDataset(DatasetId.fromPb(queryRequestPb.getDefaultDataset())); diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/QueryResponse.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/QueryResponse.java index 77386747754f..12000cc1cbd2 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/QueryResponse.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/QueryResponse.java @@ -29,20 +29,20 @@ * Query Request ({@link BigQuery#query(QueryRequest)}). * *

Example usage of a query response: - *

    {@code
- *    QueryResponse response = bigquery.query(request);
- *    while (!response.jobCompleted()) {
- *      Thread.sleep(1000);
- *      response = bigquery.getQueryResults(response.jobId());
- *    }
- *    List executionErrors = response.executionErrors();
- *    // look for errors in executionErrors
- *    QueryResult result = response.result();
- *    Iterator> rowIterator = result.iterateAll();
- *    while(rowIterator.hasNext()) {
- *      List row = rowIterator.next();
- *      // do something with row
- *    }
+ * 
 {@code
+ * QueryResponse response = bigquery.query(request);
+ * while (!response.jobCompleted()) {
+ *   Thread.sleep(1000);
+ *   response = bigquery.getQueryResults(response.jobId());
+ * }
+ * List executionErrors = response.executionErrors();
+ * // look for errors in executionErrors
+ * QueryResult result = response.result();
+ * Iterator> rowIterator = result.iterateAll();
+ * while(rowIterator.hasNext()) {
+ *   List row = rowIterator.next();
+ *   // do something with row
+ * }
  * }
* * @see Get Query diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/BigQueryRpc.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/BigQueryRpc.java similarity index 76% rename from gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/BigQueryRpc.java rename to gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/BigQueryRpc.java index 6062e19950e0..d0b740e9e390 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/BigQueryRpc.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/BigQueryRpc.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.bigquery.spi; import com.google.api.services.bigquery.model.Dataset; import com.google.api.services.bigquery.model.GetQueryResultsResponse; @@ -100,7 +100,7 @@ public Y y() { * * @throws BigQueryException upon failure */ - Dataset getDataset(String datasetId, Map options) throws BigQueryException; + Dataset getDataset(String datasetId, Map options); /** * Lists the project's datasets. Partial information is returned on a dataset (datasetReference, @@ -108,13 +108,28 @@ public Y y() { * * @throws BigQueryException upon failure */ - Tuple> listDatasets(Map options) throws BigQueryException; + Tuple> listDatasets(Map options); - Dataset create(Dataset dataset, Map options) throws BigQueryException; + /** + * Creates a new dataset. + * + * @throws BigQueryException upon failure + */ + Dataset create(Dataset dataset, Map options); - Table create(Table table, Map options) throws BigQueryException; + /** + * Creates a new table. + * + * @throws BigQueryException upon failure + */ + Table create(Table table, Map options); - Job create(Job job, Map options) throws BigQueryException; + /** + * Creates a new job. + * + * @throws BigQueryException upon failure + */ + Job create(Job job, Map options); /** * Delete the requested dataset. @@ -122,18 +137,28 @@ public Y y() { * @return {@code true} if dataset was deleted, {@code false} if it was not found * @throws BigQueryException upon failure */ - boolean deleteDataset(String datasetId, Map options) throws BigQueryException; + boolean deleteDataset(String datasetId, Map options); - Dataset patch(Dataset dataset, Map options) throws BigQueryException; + /** + * Updates dataset information. + * + * @throws BigQueryException upon failure + */ + Dataset patch(Dataset dataset, Map options); - Table patch(Table table, Map options) throws BigQueryException; + /** + * Updates table information. + * + * @throws BigQueryException upon failure + */ + Table patch(Table table, Map options); /** * Returns the requested table or {@code null} if not found. * * @throws BigQueryException upon failure */ - Table getTable(String datasetId, String tableId, Map options) throws BigQueryException; + Table getTable(String datasetId, String tableId, Map options); /** * Lists the dataset's tables. Partial information is returned on a table (tableReference, @@ -141,8 +166,7 @@ public Y y() { * * @throws BigQueryException upon failure */ - Tuple> listTables(String dataset, Map options) - throws BigQueryException; + Tuple> listTables(String dataset, Map options); /** * Delete the requested table. @@ -150,27 +174,37 @@ Tuple> listTables(String dataset, Map options * @return {@code true} if table was deleted, {@code false} if it was not found * @throws BigQueryException upon failure */ - boolean deleteTable(String datasetId, String tableId) throws BigQueryException; + boolean deleteTable(String datasetId, String tableId); + /** + * Sends an insert all request. + * + * @throws BigQueryException upon failure + */ TableDataInsertAllResponse insertAll(String datasetId, String tableId, - TableDataInsertAllRequest request) throws BigQueryException; + TableDataInsertAllRequest request); + /** + * Lists the table's rows. + * + * @throws BigQueryException upon failure + */ Tuple> listTableData(String datasetId, String tableId, - Map options) throws BigQueryException; + Map options); /** * Returns the requested job or {@code null} if not found. * * @throws BigQueryException upon failure */ - Job getJob(String jobId, Map options) throws BigQueryException; + Job getJob(String jobId, Map options); /** * Lists the project's jobs. * * @throws BigQueryException upon failure */ - Tuple> listJobs(Map options) throws BigQueryException; + Tuple> listJobs(Map options); /** * Sends a job cancel request. This call will return immediately, and the client will need to poll @@ -180,12 +214,21 @@ Tuple> listTableData(String datasetId, String tableId * found * @throws BigQueryException upon failure */ - boolean cancel(String jobId) throws BigQueryException; + boolean cancel(String jobId); - GetQueryResultsResponse getQueryResults(String jobId, Map options) - throws BigQueryException; + /** + * Returns results of the query associated with the provided job. + * + * @throws BigQueryException upon failure + */ + GetQueryResultsResponse getQueryResults(String jobId, Map options); - QueryResponse query(QueryRequest request) throws BigQueryException; + /** + * Runs the query associated with the request. + * + * @throws BigQueryException upon failure + */ + QueryResponse query(QueryRequest request); /** * Opens a resumable upload session to load data into a BigQuery table and returns an upload URI. @@ -193,7 +236,7 @@ GetQueryResultsResponse getQueryResults(String jobId, Map options) * @param configuration load configuration * @throws BigQueryException upon failure */ - String open(JobConfiguration configuration) throws BigQueryException; + String open(JobConfiguration configuration); /** * Uploads the provided data to the resumable upload session at the specified position. @@ -207,5 +250,5 @@ GetQueryResultsResponse getQueryResults(String jobId, Map options) * @throws BigQueryException upon failure */ void write(String uploadId, byte[] toWrite, int toWriteOffset, long destOffset, int length, - boolean last) throws BigQueryException; + boolean last); } diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/BigQueryRpcFactory.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/BigQueryRpcFactory.java similarity index 90% rename from gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/BigQueryRpcFactory.java rename to gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/BigQueryRpcFactory.java index 2706868756a5..1323ec0624f4 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/BigQueryRpcFactory.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/BigQueryRpcFactory.java @@ -14,9 +14,10 @@ * limitations under the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.bigquery.spi; import com.google.gcloud.bigquery.BigQueryOptions; +import com.google.gcloud.spi.ServiceRpcFactory; /** * An interface for BigQuery RPC factory. diff --git a/gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/DefaultBigQueryRpc.java b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/DefaultBigQueryRpc.java similarity index 89% rename from gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/DefaultBigQueryRpc.java rename to gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/DefaultBigQueryRpc.java index b57f1dc8a128..71712bda7806 100644 --- a/gcloud-java-bigquery/src/main/java/com/google/gcloud/spi/DefaultBigQueryRpc.java +++ b/gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/spi/DefaultBigQueryRpc.java @@ -12,14 +12,17 @@ * the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.bigquery.spi; -import static com.google.gcloud.spi.BigQueryRpc.Option.DELETE_CONTENTS; -import static com.google.gcloud.spi.BigQueryRpc.Option.FIELDS; -import static com.google.gcloud.spi.BigQueryRpc.Option.MAX_RESULTS; -import static com.google.gcloud.spi.BigQueryRpc.Option.PAGE_TOKEN; -import static com.google.gcloud.spi.BigQueryRpc.Option.START_INDEX; -import static com.google.gcloud.spi.BigQueryRpc.Option.TIMEOUT; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.ALL_DATASETS; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.ALL_USERS; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.DELETE_CONTENTS; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.FIELDS; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.MAX_RESULTS; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.PAGE_TOKEN; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.START_INDEX; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.STATE_FILTER; +import static com.google.gcloud.bigquery.spi.BigQueryRpc.Option.TIMEOUT; import static java.net.HttpURLConnection.HTTP_CREATED; import static java.net.HttpURLConnection.HTTP_NOT_FOUND; import static java.net.HttpURLConnection.HTTP_OK; @@ -90,7 +93,7 @@ private static BigQueryException translate(IOException exception) { } @Override - public Dataset getDataset(String datasetId, Map options) throws BigQueryException { + public Dataset getDataset(String datasetId, Map options) { try { return bigquery.datasets() .get(this.options.projectId(), datasetId) @@ -106,14 +109,14 @@ public Dataset getDataset(String datasetId, Map options) throws BigQu } @Override - public Tuple> listDatasets(Map options) - throws BigQueryException { + public Tuple> listDatasets(Map options) { try { DatasetList datasetsList = bigquery.datasets() .list(this.options.projectId()) - .setAll(Option.ALL_DATASETS.getBoolean(options)) + .setAll(ALL_DATASETS.getBoolean(options)) .setMaxResults(MAX_RESULTS.getLong(options)) .setPageToken(PAGE_TOKEN.getString(options)) + .setPageToken(PAGE_TOKEN.getString(options)) .execute(); Iterable datasets = datasetsList.getDatasets(); return Tuple.of(datasetsList.getNextPageToken(), @@ -135,7 +138,7 @@ public Dataset apply(DatasetList.Datasets datasetPb) { } @Override - public Dataset create(Dataset dataset, Map options) throws BigQueryException { + public Dataset create(Dataset dataset, Map options) { try { return bigquery.datasets().insert(this.options.projectId(), dataset) .setFields(FIELDS.getString(options)) @@ -146,8 +149,7 @@ public Dataset create(Dataset dataset, Map options) throws BigQueryEx } @Override - public Table create(Table table, Map options) - throws BigQueryException { + public Table create(Table table, Map options) { try { // unset the type, as it is output only table.setType(null); @@ -161,7 +163,7 @@ public Table create(Table table, Map options) } @Override - public Job create(Job job, Map options) throws BigQueryException { + public Job create(Job job, Map options) { try { return bigquery.jobs() .insert(this.options.projectId(), job) @@ -173,7 +175,7 @@ public Job create(Job job, Map options) throws BigQueryException { } @Override - public boolean deleteDataset(String datasetId, Map options) throws BigQueryException { + public boolean deleteDataset(String datasetId, Map options) { try { bigquery.datasets().delete(this.options.projectId(), datasetId) .setDeleteContents(DELETE_CONTENTS.getBoolean(options)) @@ -189,7 +191,7 @@ public boolean deleteDataset(String datasetId, Map options) throws Bi } @Override - public Dataset patch(Dataset dataset, Map options) throws BigQueryException { + public Dataset patch(Dataset dataset, Map options) { try { DatasetReference reference = dataset.getDatasetReference(); return bigquery.datasets() @@ -202,7 +204,7 @@ public Dataset patch(Dataset dataset, Map options) throws BigQueryExc } @Override - public Table patch(Table table, Map options) throws BigQueryException { + public Table patch(Table table, Map options) { try { // unset the type, as it is output only table.setType(null); @@ -217,8 +219,7 @@ public Table patch(Table table, Map options) throws BigQueryException } @Override - public Table getTable(String datasetId, String tableId, Map options) - throws BigQueryException { + public Table getTable(String datasetId, String tableId, Map options) { try { return bigquery.tables() .get(this.options.projectId(), datasetId, tableId) @@ -234,8 +235,7 @@ public Table getTable(String datasetId, String tableId, Map options) } @Override - public Tuple> listTables(String datasetId, Map options) - throws BigQueryException { + public Tuple> listTables(String datasetId, Map options) { try { TableList tableList = bigquery.tables() .list(this.options.projectId(), datasetId) @@ -262,7 +262,7 @@ public Table apply(TableList.Tables tablePb) { } @Override - public boolean deleteTable(String datasetId, String tableId) throws BigQueryException { + public boolean deleteTable(String datasetId, String tableId) { try { bigquery.tables().delete(this.options.projectId(), datasetId, tableId).execute(); return true; @@ -277,7 +277,7 @@ public boolean deleteTable(String datasetId, String tableId) throws BigQueryExce @Override public TableDataInsertAllResponse insertAll(String datasetId, String tableId, - TableDataInsertAllRequest request) throws BigQueryException { + TableDataInsertAllRequest request) { try { return bigquery.tabledata() .insertAll(this.options.projectId(), datasetId, tableId, request) @@ -289,7 +289,7 @@ public TableDataInsertAllResponse insertAll(String datasetId, String tableId, @Override public Tuple> listTableData(String datasetId, String tableId, - Map options) throws BigQueryException { + Map options) { try { TableDataList tableDataList = bigquery.tabledata() .list(this.options.projectId(), datasetId, tableId) @@ -306,7 +306,7 @@ public Tuple> listTableData(String datasetId, String } @Override - public Job getJob(String jobId, Map options) throws BigQueryException { + public Job getJob(String jobId, Map options) { try { return bigquery.jobs() .get(this.options.projectId(), jobId) @@ -322,13 +322,13 @@ public Job getJob(String jobId, Map options) throws BigQueryException } @Override - public Tuple> listJobs(Map options) throws BigQueryException { + public Tuple> listJobs(Map options) { try { JobList jobsList = bigquery.jobs() .list(this.options.projectId()) - .setAllUsers(Option.ALL_USERS.getBoolean(options)) - .setFields(Option.FIELDS.getString(options)) - .setStateFilter(Option.STATE_FILTER.>get(options)) + .setAllUsers(ALL_USERS.getBoolean(options)) + .setFields(FIELDS.getString(options)) + .setStateFilter(STATE_FILTER.>get(options)) .setMaxResults(MAX_RESULTS.getLong(options)) .setPageToken(PAGE_TOKEN.getString(options)) .setProjection(DEFAULT_PROJECTION) @@ -363,7 +363,7 @@ public Job apply(JobList.Jobs jobPb) { } @Override - public boolean cancel(String jobId) throws BigQueryException { + public boolean cancel(String jobId) { try { bigquery.jobs().cancel(this.options.projectId(), jobId).execute(); return true; @@ -377,8 +377,7 @@ public boolean cancel(String jobId) throws BigQueryException { } @Override - public GetQueryResultsResponse getQueryResults(String jobId, Map options) - throws BigQueryException { + public GetQueryResultsResponse getQueryResults(String jobId, Map options) { try { return bigquery.jobs().getQueryResults(this.options.projectId(), jobId) .setMaxResults(MAX_RESULTS.getLong(options)) @@ -397,7 +396,7 @@ public GetQueryResultsResponse getQueryResults(String jobId, Map opti } @Override - public QueryResponse query(QueryRequest request) throws BigQueryException { + public QueryResponse query(QueryRequest request) { try { return bigquery.jobs().query(this.options.projectId(), request).execute(); } catch (IOException ex) { @@ -406,7 +405,7 @@ public QueryResponse query(QueryRequest request) throws BigQueryException { } @Override - public String open(JobConfiguration configuration) throws BigQueryException { + public String open(JobConfiguration configuration) { try { Job loadJob = new Job().setConfiguration(configuration); StringBuilder builder = new StringBuilder() @@ -429,7 +428,7 @@ public String open(JobConfiguration configuration) throws BigQueryException { @Override public void write(String uploadId, byte[] toWrite, int toWriteOffset, long destOffset, int length, - boolean last) throws BigQueryException { + boolean last) { try { GenericUrl url = new GenericUrl(uploadId); HttpRequest httpRequest = bigquery.getRequestFactory().buildPutRequest(url, diff --git a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/BigQueryImplTest.java b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/BigQueryImplTest.java index 385ee6dcc8bd..a6f512800024 100644 --- a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/BigQueryImplTest.java +++ b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/BigQueryImplTest.java @@ -40,9 +40,9 @@ import com.google.gcloud.RetryParams; import com.google.gcloud.WriteChannel; import com.google.gcloud.bigquery.InsertAllRequest.RowToInsert; -import com.google.gcloud.spi.BigQueryRpc; -import com.google.gcloud.spi.BigQueryRpc.Tuple; -import com.google.gcloud.spi.BigQueryRpcFactory; +import com.google.gcloud.bigquery.spi.BigQueryRpc; +import com.google.gcloud.bigquery.spi.BigQueryRpc.Tuple; +import com.google.gcloud.bigquery.spi.BigQueryRpcFactory; import org.easymock.Capture; import org.easymock.EasyMock; @@ -148,12 +148,12 @@ public class BigQueryImplTest { private static final TableRow TABLE_ROW = new TableRow().setF(ImmutableList.of(BOOLEAN_FIELD, INTEGER_FIELD)); private static final QueryRequest QUERY_REQUEST = QueryRequest.builder("SQL") - .maxResults(42L) + .pageSize(42L) .useQueryCache(false) .defaultDataset(DatasetId.of(DATASET)) .build(); private static final QueryRequest QUERY_REQUEST_WITH_PROJECT = QueryRequest.builder("SQL") - .maxResults(42L) + .pageSize(42L) .useQueryCache(false) .defaultDataset(DatasetId.of(PROJECT, DATASET)) .build(); @@ -169,9 +169,9 @@ public class BigQueryImplTest { private static final BigQuery.DatasetListOption DATASET_LIST_ALL = BigQuery.DatasetListOption.all(); private static final BigQuery.DatasetListOption DATASET_LIST_PAGE_TOKEN = - BigQuery.DatasetListOption.startPageToken("cursor"); - private static final BigQuery.DatasetListOption DATASET_LIST_MAX_RESULTS = - BigQuery.DatasetListOption.maxResults(42L); + BigQuery.DatasetListOption.pageToken("cursor"); + private static final BigQuery.DatasetListOption DATASET_LIST_PAGE_SIZE = + BigQuery.DatasetListOption.pageSize(42L); private static final Map DATASET_LIST_OPTIONS = ImmutableMap.of( BigQueryRpc.Option.ALL_DATASETS, true, BigQueryRpc.Option.PAGE_TOKEN, "cursor", @@ -188,19 +188,19 @@ public class BigQueryImplTest { BigQuery.TableOption.fields(BigQuery.TableField.SCHEMA, BigQuery.TableField.ETAG); // Table list options - private static final BigQuery.TableListOption TABLE_LIST_MAX_RESULTS = - BigQuery.TableListOption.maxResults(42L); + private static final BigQuery.TableListOption TABLE_LIST_PAGE_SIZE = + BigQuery.TableListOption.pageSize(42L); private static final BigQuery.TableListOption TABLE_LIST_PAGE_TOKEN = - BigQuery.TableListOption.startPageToken("cursor"); + BigQuery.TableListOption.pageToken("cursor"); private static final Map TABLE_LIST_OPTIONS = ImmutableMap.of( BigQueryRpc.Option.MAX_RESULTS, 42L, BigQueryRpc.Option.PAGE_TOKEN, "cursor"); // TableData list options - private static final BigQuery.TableDataListOption TABLE_DATA_LIST_MAX_RESULTS = - BigQuery.TableDataListOption.maxResults(42L); + private static final BigQuery.TableDataListOption TABLE_DATA_LIST_PAGE_SIZE = + BigQuery.TableDataListOption.pageSize(42L); private static final BigQuery.TableDataListOption TABLE_DATA_LIST_PAGE_TOKEN = - BigQuery.TableDataListOption.startPageToken("cursor"); + BigQuery.TableDataListOption.pageToken("cursor"); private static final BigQuery.TableDataListOption TABLE_DATA_LIST_START_INDEX = BigQuery.TableDataListOption.startIndex(0L); private static final Map TABLE_DATA_LIST_OPTIONS = ImmutableMap.of( @@ -220,9 +220,9 @@ public class BigQueryImplTest { private static final BigQuery.JobListOption JOB_LIST_STATE_FILTER = BigQuery.JobListOption.stateFilter(JobStatus.State.DONE, JobStatus.State.PENDING); private static final BigQuery.JobListOption JOB_LIST_PAGE_TOKEN = - BigQuery.JobListOption.startPageToken("cursor"); - private static final BigQuery.JobListOption JOB_LIST_MAX_RESULTS = - BigQuery.JobListOption.maxResults(42L); + BigQuery.JobListOption.pageToken("cursor"); + private static final BigQuery.JobListOption JOB_LIST_PAGE_SIZE = + BigQuery.JobListOption.pageSize(42L); private static final Map JOB_LIST_OPTIONS = ImmutableMap.of( BigQueryRpc.Option.ALL_USERS, true, BigQueryRpc.Option.STATE_FILTER, ImmutableList.of("done", "pending"), @@ -235,9 +235,9 @@ public class BigQueryImplTest { private static final BigQuery.QueryResultsOption QUERY_RESULTS_OPTION_INDEX = BigQuery.QueryResultsOption.startIndex(1024L); private static final BigQuery.QueryResultsOption QUERY_RESULTS_OPTION_PAGE_TOKEN = - BigQuery.QueryResultsOption.startPageToken("cursor"); - private static final BigQuery.QueryResultsOption QUERY_RESULTS_OPTION_MAX_RESULTS = - BigQuery.QueryResultsOption.maxResults(0L); + BigQuery.QueryResultsOption.pageToken("cursor"); + private static final BigQuery.QueryResultsOption QUERY_RESULTS_OPTION_PAGE_SIZE = + BigQuery.QueryResultsOption.pageSize(0L); private static final Map QUERY_RESULTS_OPTIONS = ImmutableMap.of( BigQueryRpc.Option.TIMEOUT, 42L, BigQueryRpc.Option.START_INDEX, 1024L, @@ -388,7 +388,7 @@ public void testListDatasetsWithOptions() { EasyMock.expect(bigqueryRpcMock.listDatasets(DATASET_LIST_OPTIONS)).andReturn(result); EasyMock.replay(bigqueryRpcMock); Page page = bigquery.listDatasets(DATASET_LIST_ALL, DATASET_LIST_PAGE_TOKEN, - DATASET_LIST_MAX_RESULTS); + DATASET_LIST_PAGE_SIZE); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(datasetList.toArray(), Iterables.toArray(page.values(), DatasetInfo.class)); } @@ -560,7 +560,7 @@ public void testListTablesWithOptions() { Tuple.of(cursor, Iterables.transform(tableList, TableInfo.TO_PB_FUNCTION)); EasyMock.expect(bigqueryRpcMock.listTables(DATASET, TABLE_LIST_OPTIONS)).andReturn(result); EasyMock.replay(bigqueryRpcMock); - Page
page = bigquery.listTables(DATASET, TABLE_LIST_MAX_RESULTS, TABLE_LIST_PAGE_TOKEN); + Page
page = bigquery.listTables(DATASET, TABLE_LIST_PAGE_SIZE, TABLE_LIST_PAGE_TOKEN); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(tableList.toArray(), Iterables.toArray(page.values(), Table.class)); } @@ -733,7 +733,7 @@ public void testListTableDataWithOptions() { EasyMock.replay(bigqueryRpcMock); bigquery = options.service(); Page> page = bigquery.listTableData(DATASET, TABLE, - TABLE_DATA_LIST_MAX_RESULTS, TABLE_DATA_LIST_PAGE_TOKEN, TABLE_DATA_LIST_START_INDEX); + TABLE_DATA_LIST_PAGE_SIZE, TABLE_DATA_LIST_PAGE_TOKEN, TABLE_DATA_LIST_START_INDEX); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(tableData.toArray(), Iterables.toArray(page.values(), List.class)); } @@ -859,7 +859,7 @@ public com.google.api.services.bigquery.model.Job apply(Job job) { EasyMock.expect(bigqueryRpcMock.listJobs(JOB_LIST_OPTIONS)).andReturn(result); EasyMock.replay(bigqueryRpcMock); Page page = bigquery.listJobs(JOB_LIST_ALL_USERS, JOB_LIST_STATE_FILTER, - JOB_LIST_PAGE_TOKEN, JOB_LIST_MAX_RESULTS); + JOB_LIST_PAGE_TOKEN, JOB_LIST_PAGE_SIZE); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(jobList.toArray(), Iterables.toArray(page.values(), Job.class)); } @@ -1012,7 +1012,7 @@ public void testGetQueryResultsWithOptions() { EasyMock.replay(bigqueryRpcMock); bigquery = options.service(); QueryResponse response = bigquery.getQueryResults(queryJob, QUERY_RESULTS_OPTION_TIME, - QUERY_RESULTS_OPTION_INDEX, QUERY_RESULTS_OPTION_MAX_RESULTS, + QUERY_RESULTS_OPTION_INDEX, QUERY_RESULTS_OPTION_PAGE_SIZE, QUERY_RESULTS_OPTION_PAGE_TOKEN); assertEquals(queryJob, response.jobId()); assertEquals(true, response.jobCompleted()); diff --git a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/DatasetTest.java b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/DatasetTest.java index 373291021b23..dd03b7899ebc 100644 --- a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/DatasetTest.java +++ b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/DatasetTest.java @@ -260,11 +260,11 @@ public void testListWithOptions() throws Exception { new Table(serviceMockReturnsOptions, new Table.BuilderImpl(TABLE_INFO3))); PageImpl
expectedPage = new PageImpl<>(null, "c", tableResults); expect(bigquery.options()).andReturn(mockOptions); - expect(bigquery.listTables(DATASET_INFO.datasetId(), BigQuery.TableListOption.maxResults(10L))) + expect(bigquery.listTables(DATASET_INFO.datasetId(), BigQuery.TableListOption.pageSize(10L))) .andReturn(expectedPage); replay(bigquery); initializeDataset(); - Page
tablePage = dataset.list(BigQuery.TableListOption.maxResults(10L)); + Page
tablePage = dataset.list(BigQuery.TableListOption.pageSize(10L)); assertArrayEquals(tableResults.toArray(), Iterables.toArray(tablePage.values(), Table.class)); assertEquals(expectedPage.nextPageCursor(), tablePage.nextPageCursor()); } diff --git a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/OptionTest.java b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/OptionTest.java index 225fc284b203..2c89ececedb8 100644 --- a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/OptionTest.java +++ b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/OptionTest.java @@ -18,7 +18,7 @@ import static org.junit.Assert.assertEquals; -import com.google.gcloud.spi.BigQueryRpc; +import com.google.gcloud.bigquery.spi.BigQueryRpc; import org.junit.Test; diff --git a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/QueryRequestTest.java b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/QueryRequestTest.java index 370b4d614cbf..7875dee9e315 100644 --- a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/QueryRequestTest.java +++ b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/QueryRequestTest.java @@ -29,13 +29,13 @@ public class QueryRequestTest { private static final DatasetId DATASET_ID = DatasetId.of("dataset"); private static final Boolean USE_QUERY_CACHE = true; private static final Boolean DRY_RUN = false; - private static final Long MAX_RESULTS = 42L; + private static final Long PAGE_SIZE = 42L; private static final Long MAX_WAIT_TIME = 42000L; private static final QueryRequest QUERY_REQUEST = QueryRequest.builder(QUERY) .useQueryCache(USE_QUERY_CACHE) .defaultDataset(DATASET_ID) .dryRun(DRY_RUN) - .maxResults(MAX_RESULTS) + .pageSize(PAGE_SIZE) .maxWaitTime(MAX_WAIT_TIME) .build(); @@ -65,7 +65,7 @@ public void testBuilder() { assertEquals(USE_QUERY_CACHE, QUERY_REQUEST.useQueryCache()); assertEquals(DATASET_ID, QUERY_REQUEST.defaultDataset()); assertEquals(DRY_RUN, QUERY_REQUEST.dryRun()); - assertEquals(MAX_RESULTS, QUERY_REQUEST.maxResults()); + assertEquals(PAGE_SIZE, QUERY_REQUEST.pageSize()); assertEquals(MAX_WAIT_TIME, QUERY_REQUEST.maxWaitTime()); thrown.expect(NullPointerException.class); QueryRequest.builder(null); @@ -78,7 +78,7 @@ public void testOf() { assertNull(request.useQueryCache()); assertNull(request.defaultDataset()); assertNull(request.dryRun()); - assertNull(request.maxResults()); + assertNull(request.pageSize()); assertNull(request.maxWaitTime()); thrown.expect(NullPointerException.class); QueryRequest.of(null); @@ -102,7 +102,7 @@ private void compareQueryRequest(QueryRequest expected, QueryRequest value) { assertEquals(expected.useQueryCache(), value.useQueryCache()); assertEquals(expected.defaultDataset(), value.defaultDataset()); assertEquals(expected.dryRun(), value.dryRun()); - assertEquals(expected.maxResults(), value.maxResults()); + assertEquals(expected.pageSize(), value.pageSize()); assertEquals(expected.maxWaitTime(), value.maxWaitTime()); } } diff --git a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/SerializationTest.java b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/SerializationTest.java index d877bff2138c..111df074ffa2 100644 --- a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/SerializationTest.java +++ b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/SerializationTest.java @@ -16,30 +16,19 @@ package com.google.gcloud.bigquery; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotSame; - import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import com.google.gcloud.AuthCredentials; -import com.google.gcloud.RestorableState; -import com.google.gcloud.RetryParams; -import com.google.gcloud.WriteChannel; +import com.google.gcloud.BaseSerializationTest; +import com.google.gcloud.Restorable; import com.google.gcloud.bigquery.StandardTableDefinition.StreamingBuffer; -import org.junit.Test; - -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.IOException; -import java.io.ObjectInputStream; -import java.io.ObjectOutputStream; import java.io.Serializable; import java.nio.charset.StandardCharsets; import java.util.List; import java.util.Map; -public class SerializationTest { +public class SerializationTest extends BaseSerializationTest { private static final Acl DOMAIN_ACCESS = Acl.of(new Acl.Domain("domain"), Acl.Role.WRITER); @@ -207,7 +196,7 @@ public class SerializationTest { .useQueryCache(true) .defaultDataset(DATASET_ID) .dryRun(false) - .maxResults(42L) + .pageSize(42L) .maxWaitTime(10L) .build(); private static final QueryResult QUERY_RESULT = QueryResult.builder() @@ -230,75 +219,40 @@ public class SerializationTest { new Dataset(BIGQUERY, new DatasetInfo.BuilderImpl(DATASET_INFO)); private static final Table TABLE = new Table(BIGQUERY, new TableInfo.BuilderImpl(TABLE_INFO)); private static final Job JOB = new Job(BIGQUERY, new JobInfo.BuilderImpl(JOB_INFO)); + private static final BigQueryException BIG_QUERY_EXCEPTION = + new BigQueryException(42, "message", BIGQUERY_ERROR); - @Test - public void testServiceOptions() throws Exception { + @Override + protected Serializable[] serializableObjects() { BigQueryOptions options = BigQueryOptions.builder() .projectId("p1") .authCredentials(AuthCredentials.createForAppEngine()) .build(); - BigQueryOptions serializedCopy = serializeAndDeserialize(options); - assertEquals(options, serializedCopy); - - options = options.toBuilder() + BigQueryOptions otherOptions = options.toBuilder() .projectId("p2") - .retryParams(RetryParams.defaultInstance()) .authCredentials(null) .build(); - serializedCopy = serializeAndDeserialize(options); - assertEquals(options, serializedCopy); - } - - @Test - public void testModelAndRequests() throws Exception { - Serializable[] objects = {DOMAIN_ACCESS, GROUP_ACCESS, USER_ACCESS, VIEW_ACCESS, DATASET_ID, + return new Serializable[]{DOMAIN_ACCESS, GROUP_ACCESS, USER_ACCESS, VIEW_ACCESS, DATASET_ID, DATASET_INFO, TABLE_ID, CSV_OPTIONS, STREAMING_BUFFER, TABLE_DEFINITION, EXTERNAL_TABLE_DEFINITION, VIEW_DEFINITION, TABLE_SCHEMA, TABLE_INFO, VIEW_INFO, EXTERNAL_TABLE_INFO, INLINE_FUNCTION, URI_FUNCTION, JOB_STATISTICS, EXTRACT_STATISTICS, LOAD_STATISTICS, QUERY_STATISTICS, BIGQUERY_ERROR, JOB_STATUS, JOB_ID, COPY_JOB_CONFIGURATION, EXTRACT_JOB_CONFIGURATION, LOAD_CONFIGURATION, LOAD_JOB_CONFIGURATION, QUERY_JOB_CONFIGURATION, JOB_INFO, INSERT_ALL_REQUEST, - INSERT_ALL_RESPONSE, FIELD_VALUE, QUERY_REQUEST, QUERY_RESPONSE, + INSERT_ALL_RESPONSE, FIELD_VALUE, QUERY_REQUEST, QUERY_RESPONSE, BIG_QUERY_EXCEPTION, BigQuery.DatasetOption.fields(), BigQuery.DatasetDeleteOption.deleteContents(), BigQuery.DatasetListOption.all(), BigQuery.TableOption.fields(), - BigQuery.TableListOption.maxResults(42L), BigQuery.JobOption.fields(), - BigQuery.JobListOption.allUsers(), DATASET, TABLE, JOB}; - for (Serializable obj : objects) { - Object copy = serializeAndDeserialize(obj); - assertEquals(obj, obj); - assertEquals(obj, copy); - assertNotSame(obj, copy); - assertEquals(copy, copy); - } + BigQuery.TableListOption.pageSize(42L), BigQuery.JobOption.fields(), + BigQuery.JobListOption.allUsers(), DATASET, TABLE, JOB, options, otherOptions}; } - @Test - public void testWriteChannelState() throws IOException, ClassNotFoundException { - BigQueryOptions options = BigQueryOptions.builder() - .projectId("p2") - .retryParams(RetryParams.defaultInstance()) - .build(); + @Override + protected Restorable[] restorableObjects() { + BigQueryOptions options = BigQueryOptions.builder().projectId("p2").build(); // avoid closing when you don't want partial writes upon failure @SuppressWarnings("resource") TableDataWriteChannel writer = new TableDataWriteChannel(options, LOAD_CONFIGURATION, "upload-id"); - RestorableState state = writer.capture(); - RestorableState deserializedState = serializeAndDeserialize(state); - assertEquals(state, deserializedState); - assertEquals(state.hashCode(), deserializedState.hashCode()); - assertEquals(state.toString(), deserializedState.toString()); - } - - @SuppressWarnings("unchecked") - private T serializeAndDeserialize(T obj) - throws IOException, ClassNotFoundException { - ByteArrayOutputStream bytes = new ByteArrayOutputStream(); - try (ObjectOutputStream output = new ObjectOutputStream(bytes)) { - output.writeObject(obj); - } - try (ObjectInputStream input = - new ObjectInputStream(new ByteArrayInputStream(bytes.toByteArray()))) { - return (T) input.readObject(); - } + return new Restorable[]{writer}; } } diff --git a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/TableDataWriteChannelTest.java b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/TableDataWriteChannelTest.java index 6b7edcd76db1..4c1be470ff57 100644 --- a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/TableDataWriteChannelTest.java +++ b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/TableDataWriteChannelTest.java @@ -32,8 +32,8 @@ import com.google.gcloud.RestorableState; import com.google.gcloud.WriteChannel; -import com.google.gcloud.spi.BigQueryRpc; -import com.google.gcloud.spi.BigQueryRpcFactory; +import com.google.gcloud.bigquery.spi.BigQueryRpc; +import com.google.gcloud.bigquery.spi.BigQueryRpcFactory; import org.easymock.Capture; import org.easymock.CaptureType; diff --git a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/TableTest.java b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/TableTest.java index 4866ee9ab8ec..c7828ebeadf4 100644 --- a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/TableTest.java +++ b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/TableTest.java @@ -286,11 +286,11 @@ public void testListWithOptions() throws Exception { initializeExpectedTable(1); expect(bigquery.options()).andReturn(mockOptions); PageImpl> tableDataPage = new PageImpl<>(null, "c", ROWS); - expect(bigquery.listTableData(TABLE_ID1, BigQuery.TableDataListOption.maxResults(10L))) + expect(bigquery.listTableData(TABLE_ID1, BigQuery.TableDataListOption.pageSize(10L))) .andReturn(tableDataPage); replay(bigquery); initializeTable(); - Page> dataPage = table.list(BigQuery.TableDataListOption.maxResults(10L)); + Page> dataPage = table.list(BigQuery.TableDataListOption.pageSize(10L)); Iterator> tableDataIterator = tableDataPage.values().iterator(); Iterator> dataIterator = dataPage.values().iterator(); assertTrue(Iterators.elementsEqual(tableDataIterator, dataIterator)); diff --git a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/ITBigQueryTest.java b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/it/ITBigQueryTest.java similarity index 93% rename from gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/ITBigQueryTest.java rename to gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/it/ITBigQueryTest.java index 8021809be6b2..50780b4fc9a9 100644 --- a/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/ITBigQueryTest.java +++ b/gcloud-java-bigquery/src/test/java/com/google/gcloud/bigquery/it/ITBigQueryTest.java @@ -14,14 +14,8 @@ * limitations under the License. */ -package com.google.gcloud.bigquery; - -import static com.google.gcloud.bigquery.BigQuery.DatasetField; -import static com.google.gcloud.bigquery.BigQuery.JobField; -import static com.google.gcloud.bigquery.BigQuery.JobListOption; -import static com.google.gcloud.bigquery.BigQuery.JobOption; -import static com.google.gcloud.bigquery.BigQuery.TableField; -import static com.google.gcloud.bigquery.BigQuery.TableOption; +package com.google.gcloud.bigquery.it; + import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNotNull; @@ -32,7 +26,43 @@ import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import com.google.gcloud.Page; +import com.google.gcloud.WriteChannel; +import com.google.gcloud.bigquery.BigQuery; +import com.google.gcloud.bigquery.BigQuery.DatasetField; import com.google.gcloud.bigquery.BigQuery.DatasetOption; +import com.google.gcloud.bigquery.BigQuery.JobField; +import com.google.gcloud.bigquery.BigQuery.JobListOption; +import com.google.gcloud.bigquery.BigQuery.JobOption; +import com.google.gcloud.bigquery.BigQuery.TableField; +import com.google.gcloud.bigquery.BigQuery.TableOption; +import com.google.gcloud.bigquery.BigQueryError; +import com.google.gcloud.bigquery.BigQueryException; +import com.google.gcloud.bigquery.CopyJobConfiguration; +import com.google.gcloud.bigquery.Dataset; +import com.google.gcloud.bigquery.DatasetId; +import com.google.gcloud.bigquery.DatasetInfo; +import com.google.gcloud.bigquery.ExternalTableDefinition; +import com.google.gcloud.bigquery.ExtractJobConfiguration; +import com.google.gcloud.bigquery.Field; +import com.google.gcloud.bigquery.FieldValue; +import com.google.gcloud.bigquery.FormatOptions; +import com.google.gcloud.bigquery.InsertAllRequest; +import com.google.gcloud.bigquery.InsertAllResponse; +import com.google.gcloud.bigquery.Job; +import com.google.gcloud.bigquery.JobInfo; +import com.google.gcloud.bigquery.JobStatistics; +import com.google.gcloud.bigquery.LoadJobConfiguration; +import com.google.gcloud.bigquery.QueryJobConfiguration; +import com.google.gcloud.bigquery.QueryRequest; +import com.google.gcloud.bigquery.QueryResponse; +import com.google.gcloud.bigquery.Schema; +import com.google.gcloud.bigquery.StandardTableDefinition; +import com.google.gcloud.bigquery.Table; +import com.google.gcloud.bigquery.TableDefinition; +import com.google.gcloud.bigquery.TableId; +import com.google.gcloud.bigquery.TableInfo; +import com.google.gcloud.bigquery.ViewDefinition; +import com.google.gcloud.bigquery.WriteChannelConfiguration; import com.google.gcloud.bigquery.testing.RemoteBigQueryHelper; import com.google.gcloud.storage.BlobInfo; import com.google.gcloud.storage.BucketInfo; @@ -45,7 +75,6 @@ import org.junit.Test; import org.junit.rules.Timeout; -import java.io.FileNotFoundException; import java.io.IOException; import java.nio.ByteBuffer; import java.nio.charset.StandardCharsets; @@ -140,7 +169,7 @@ public class ITBigQueryTest { public Timeout globalTimeout = Timeout.seconds(300); @BeforeClass - public static void beforeClass() throws IOException, InterruptedException { + public static void beforeClass() throws InterruptedException { RemoteBigQueryHelper bigqueryHelper = RemoteBigQueryHelper.create(); RemoteGcsHelper gcsHelper = RemoteGcsHelper.create(); bigquery = bigqueryHelper.options().service(); @@ -169,8 +198,9 @@ public static void afterClass() throws ExecutionException, InterruptedException if (bigquery != null) { RemoteBigQueryHelper.forceDelete(bigquery, DATASET); } - if (storage != null && !RemoteGcsHelper.forceDelete(storage, BUCKET, 10, TimeUnit.SECONDS)) { - if (LOG.isLoggable(Level.WARNING)) { + if (storage != null) { + boolean wasDeleted = RemoteGcsHelper.forceDelete(storage, BUCKET, 10, TimeUnit.SECONDS); + if (!wasDeleted && LOG.isLoggable(Level.WARNING)) { LOG.log(Level.WARNING, "Deletion of bucket {0} timed out, bucket is not empty", BUCKET); } } @@ -318,7 +348,7 @@ public void testCreateExternalTable() throws InterruptedException { + tableName) .defaultDataset(DatasetId.of(DATASET)) .maxWaitTime(60000L) - .maxResults(1000L) + .pageSize(1000L) .build(); QueryResponse response = bigquery.query(request); while (!response.jobCompleted()) { @@ -381,7 +411,7 @@ public void testCreateViewTable() throws InterruptedException { QueryRequest request = QueryRequest.builder("SELECT * FROM " + tableName) .defaultDataset(DatasetId.of(DATASET)) .maxWaitTime(60000L) - .maxResults(1000L) + .pageSize(1000L) .build(); QueryResponse response = bigquery.query(request); while (!response.jobCompleted()) { @@ -632,7 +662,7 @@ public void testQuery() throws InterruptedException { QueryRequest request = QueryRequest.builder(query) .defaultDataset(DatasetId.of(DATASET)) .maxWaitTime(60000L) - .maxResults(1000L) + .pageSize(1000L) .build(); QueryResponse response = bigquery.query(request); while (!response.jobCompleted()) { @@ -654,10 +684,9 @@ public void testQuery() throws InterruptedException { rowCount++; } assertEquals(2, rowCount); - // todo(mziccard) uncomment as soon as #624 is closed - // Job queryJob = bigquery.getJob(response.jobId()); - // JobStatistics.QueryStatistics statistics = queryJob.statistics(); - // assertNotNull(statistics.queryPlan()); + Job queryJob = bigquery.getJob(response.jobId()); + JobStatistics.QueryStatistics statistics = queryJob.statistics(); + assertNotNull(statistics.queryPlan()); } @Test @@ -685,7 +714,7 @@ public void testListJobsWithSelectedFields() { } @Test - public void testCreateAndGetJob() throws InterruptedException { + public void testCreateAndGetJob() { String sourceTableName = "test_create_and_get_job_source_table"; String destinationTableName = "test_create_and_get_job_destination_table"; TableId sourceTable = TableId.of(DATASET, sourceTableName); @@ -717,7 +746,7 @@ public void testCreateAndGetJob() throws InterruptedException { } @Test - public void testCreateAndGetJobWithSelectedFields() throws InterruptedException { + public void testCreateAndGetJobWithSelectedFields() { String sourceTableName = "test_create_and_get_job_with_selected_fields_source_table"; String destinationTableName = "test_create_and_get_job_with_selected_fields_destination_table"; TableId sourceTable = TableId.of(DATASET, sourceTableName); @@ -822,10 +851,9 @@ public void testQueryJob() throws InterruptedException { } assertEquals(2, rowCount); assertTrue(bigquery.delete(DATASET, tableName)); - // todo(mziccard) uncomment as soon as #624 is closed - // Job queryJob = bigquery.getJob(remoteJob.jobId()); - // JobStatistics.QueryStatistics statistics = queryJob.statistics(); - // assertNotNull(statistics.queryPlan()); + Job queryJob = bigquery.getJob(remoteJob.jobId()); + JobStatistics.QueryStatistics statistics = queryJob.statistics(); + assertNotNull(statistics.queryPlan()); } @Test @@ -874,12 +902,12 @@ public void testCancelJob() throws InterruptedException { } @Test - public void testCancelNonExistingJob() throws InterruptedException { + public void testCancelNonExistingJob() { assertFalse(bigquery.cancel("test_cancel_non_existing_job")); } @Test - public void testInsertFromFile() throws InterruptedException, FileNotFoundException { + public void testInsertFromFile() throws InterruptedException { String destinationTableName = "test_insert_from_file_table"; TableId tableId = TableId.of(DATASET, destinationTableName); WriteChannelConfiguration configuration = WriteChannelConfiguration.builder(tableId) @@ -887,7 +915,7 @@ public void testInsertFromFile() throws InterruptedException, FileNotFoundExcept .createDisposition(JobInfo.CreateDisposition.CREATE_IF_NEEDED) .schema(TABLE_SCHEMA) .build(); - try (TableDataWriteChannel channel = bigquery.writer(configuration)) { + try (WriteChannel channel = bigquery.writer(configuration)) { channel.write(ByteBuffer.wrap(JSON_CONTENT.getBytes(StandardCharsets.UTF_8))); } catch (IOException e) { fail("IOException was not expected"); diff --git a/gcloud-java-contrib/README.md b/gcloud-java-contrib/README.md index 23713f7450a3..426417d54e87 100644 --- a/gcloud-java-contrib/README.md +++ b/gcloud-java-contrib/README.md @@ -3,6 +3,12 @@ Google Cloud Java Contributions Packages that provide higher-level abstraction/functionality for common gcloud-java use cases. +[![Build Status](https://travis-ci.org/GoogleCloudPlatform/gcloud-java.svg?branch=master)](https://travis-ci.org/GoogleCloudPlatform/gcloud-java) +[![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) +[![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-bigquery.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-bigquery.svg) +[![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) + Quickstart ---------- If you are using Maven, add this to your pom.xml file @@ -10,16 +16,16 @@ If you are using Maven, add this to your pom.xml file com.google.gcloud gcloud-java-contrib - 0.1.3 + 0.1.5 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.gcloud:gcloud-java-contrib:0.1.3' +compile 'com.google.gcloud:gcloud-java-contrib:0.1.5' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.gcloud" % "gcloud-java-contrib" % "0.1.3" +libraryDependencies += "com.google.gcloud" % "gcloud-java-contrib" % "0.1.5" ``` Java Versions diff --git a/gcloud-java-contrib/pom.xml b/gcloud-java-contrib/pom.xml index 35dfa606ae42..bd4a6458dc38 100644 --- a/gcloud-java-contrib/pom.xml +++ b/gcloud-java-contrib/pom.xml @@ -10,7 +10,7 @@ com.google.gcloud gcloud-java-pom - 0.1.4-SNAPSHOT + 0.1.6-SNAPSHOT gcloud-java-contrib diff --git a/gcloud-java-core/README.md b/gcloud-java-core/README.md index 8b91e49f3860..fc5f481f8ec3 100644 --- a/gcloud-java-core/README.md +++ b/gcloud-java-core/README.md @@ -7,6 +7,7 @@ This module provides common functionality required by service-specific modules o [![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) [![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-core.svg)](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-core.svg) [![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) - [Homepage] (https://googlecloudplatform.github.io/gcloud-java/) - [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/package-summary.html) @@ -18,16 +19,16 @@ If you are using Maven, add this to your pom.xml file com.google.gcloud gcloud-java-core - 0.1.3 + 0.1.5 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.gcloud:gcloud-java-core:0.1.3' +compile 'com.google.gcloud:gcloud-java-core:0.1.5' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.gcloud" % "gcloud-java-core" % "0.1.3" +libraryDependencies += "com.google.gcloud" % "gcloud-java-core" % "0.1.5" ``` Troubleshooting diff --git a/gcloud-java-core/pom.xml b/gcloud-java-core/pom.xml index bd7f26e0bc3b..6d0ed675b423 100644 --- a/gcloud-java-core/pom.xml +++ b/gcloud-java-core/pom.xml @@ -10,7 +10,7 @@ com.google.gcloud gcloud-java-pom - 0.1.4-SNAPSHOT + 0.1.6-SNAPSHOT gcloud-java-core @@ -35,24 +35,24 @@ com.google.http-client google-http-client - 1.20.0 + 1.21.0 compile com.google.oauth-client google-oauth-client - 1.20.0 + 1.21.0 compile com.google.guava guava - 18.0 + 19.0 com.google.api-client google-api-client-appengine - 1.20.0 + 1.21.0 compile @@ -64,7 +64,7 @@ com.google.http-client google-http-client-jackson - 1.20.0 + 1.21.0 compile @@ -82,19 +82,19 @@ joda-time joda-time - 2.8.2 + 2.9.2 compile org.json json - 20090211 + 20151123 compile org.easymock easymock - 3.3 + 3.4 test diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/AuthCredentials.java b/gcloud-java-core/src/main/java/com/google/gcloud/AuthCredentials.java index fc5d74d0896c..27cafc181505 100644 --- a/gcloud-java-core/src/main/java/com/google/gcloud/AuthCredentials.java +++ b/gcloud-java-core/src/main/java/com/google/gcloud/AuthCredentials.java @@ -132,6 +132,12 @@ public RestorableState capture() { } } + /** + * Represents service account credentials. + * + * @see + * User accounts and service accounts + */ public static class ServiceAccountAuthCredentials extends AuthCredentials { private final String account; @@ -195,6 +201,14 @@ public RestorableState capture() { } } + /** + * Represents Application Default Credentials, which are credentials that are inferred from the + * runtime environment. + * + * @see + * Google Application Default Credentials + */ public static class ApplicationDefaultAuthCredentials extends AuthCredentials { private GoogleCredentials googleCredentials; @@ -243,6 +257,50 @@ public RestorableState capture() { } } + /** + * A placeholder for credentials to signify that requests sent to the server should not be + * authenticated. This is typically useful when using the local service emulators, such as + * {@code LocalGcdHelper} and {@code LocalResourceManagerHelper}. + */ + public static class NoAuthCredentials extends AuthCredentials { + + private static final AuthCredentials INSTANCE = new NoAuthCredentials(); + private static final NoAuthCredentialsState STATE = new NoAuthCredentialsState(); + + private static class NoAuthCredentialsState + implements RestorableState, Serializable { + + private static final long serialVersionUID = -4022100563954640465L; + + @Override + public AuthCredentials restore() { + return INSTANCE; + } + + @Override + public int hashCode() { + return getClass().getName().hashCode(); + } + + @Override + public boolean equals(Object obj) { + return obj instanceof NoAuthCredentialsState; + } + } + + private NoAuthCredentials() {} + + @Override + public GoogleCredentials credentials() { + return null; + } + + @Override + public RestorableState capture() { + return STATE; + } + } + public abstract GoogleCredentials credentials(); public static AuthCredentials createForAppEngine() { @@ -281,6 +339,15 @@ public static ServiceAccountAuthCredentials createFor(String account, PrivateKey return new ServiceAccountAuthCredentials(account, privateKey); } + /** + * Creates a placeholder denoting that no credentials should be used. This is typically useful + * when using the local service emulators, such as {@code LocalGcdHelper} and + * {@code LocalResourceManagerHelper}. + */ + public static AuthCredentials noAuth() { + return NoAuthCredentials.INSTANCE; + } + /** * Creates Service Account Credentials given a stream for credentials in JSON format. * diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/BaseServiceException.java b/gcloud-java-core/src/main/java/com/google/gcloud/BaseServiceException.java index 579340f1256e..4e0d03e0073a 100644 --- a/gcloud-java-core/src/main/java/com/google/gcloud/BaseServiceException.java +++ b/gcloud-java-core/src/main/java/com/google/gcloud/BaseServiceException.java @@ -32,6 +32,16 @@ */ public class BaseServiceException extends RuntimeException { + private static final long serialVersionUID = 759921776378760835L; + public static final int UNKNOWN_CODE = 0; + + private final int code; + private final boolean retryable; + private final String reason; + private final boolean idempotent; + private final String location; + private final String debugInfo; + protected static final class Error implements Serializable { private static final long serialVersionUID = -4019600198652965721L; @@ -79,16 +89,6 @@ public int hashCode() { } } - private static final long serialVersionUID = 759921776378760835L; - public static final int UNKNOWN_CODE = 0; - - private final int code; - private final boolean retryable; - private final String reason; - private final boolean idempotent; - private final String location; - private final String debugInfo; - public BaseServiceException(IOException exception, boolean idempotent) { super(message(exception), exception); int code = UNKNOWN_CODE; @@ -97,13 +97,17 @@ public BaseServiceException(IOException exception, boolean idempotent) { String debugInfo = null; if (exception instanceof GoogleJsonResponseException) { GoogleJsonError jsonError = ((GoogleJsonResponseException) exception).getDetails(); - Error error = error(jsonError); - code = error.code; - reason = error.reason; - if (reason != null) { - GoogleJsonError.ErrorInfo errorInfo = jsonError.getErrors().get(0); - location = errorInfo.getLocation(); - debugInfo = (String) errorInfo.get("debugInfo"); + if (jsonError != null) { + Error error = error(jsonError); + code = error.code; + reason = error.reason; + if (reason != null) { + GoogleJsonError.ErrorInfo errorInfo = jsonError.getErrors().get(0); + location = errorInfo.getLocation(); + debugInfo = (String) errorInfo.get("debugInfo"); + } + } else { + code = ((GoogleJsonResponseException) exception).getStatusCode(); } } this.code = code; @@ -194,6 +198,31 @@ protected String debugInfo() { return debugInfo; } + @Override + public boolean equals(Object obj) { + if (obj == this) { + return true; + } + if (!(obj instanceof BaseServiceException)) { + return false; + } + BaseServiceException other = (BaseServiceException) obj; + return Objects.equals(getCause(), other.getCause()) + && Objects.equals(getMessage(), other.getMessage()) + && code == other.code + && retryable == other.retryable + && Objects.equals(reason, other.reason) + && idempotent == other.idempotent + && Objects.equals(location, other.location) + && Objects.equals(debugInfo, other.debugInfo); + } + + @Override + public int hashCode() { + return Objects.hash(getCause(), getMessage(), code, retryable, reason, idempotent, location, + debugInfo); + } + protected static String reason(GoogleJsonError error) { if (error.getErrors() != null && !error.getErrors().isEmpty()) { return error.getErrors().get(0).getReason(); @@ -207,7 +236,10 @@ protected static Error error(GoogleJsonError error) { protected static String message(IOException exception) { if (exception instanceof GoogleJsonResponseException) { - return ((GoogleJsonResponseException) exception).getDetails().getMessage(); + GoogleJsonError details = ((GoogleJsonResponseException) exception).getDetails(); + if (details != null) { + return details.getMessage(); + } } return exception.getMessage(); } diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/BaseWriteChannel.java b/gcloud-java-core/src/main/java/com/google/gcloud/BaseWriteChannel.java index e05383a65826..1d18a5a27e81 100644 --- a/gcloud-java-core/src/main/java/com/google/gcloud/BaseWriteChannel.java +++ b/gcloud-java-core/src/main/java/com/google/gcloud/BaseWriteChannel.java @@ -21,6 +21,7 @@ import java.io.IOException; import java.io.Serializable; import java.nio.ByteBuffer; +import java.nio.channels.ClosedChannelException; import java.util.Arrays; import java.util.Objects; @@ -114,9 +115,9 @@ private void flush() { } } - private void validateOpen() throws IOException { + private void validateOpen() throws ClosedChannelException { if (!isOpen) { - throw new IOException("stream is closed"); + throw new ClosedChannelException(); } } diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/ExceptionHandler.java b/gcloud-java-core/src/main/java/com/google/gcloud/ExceptionHandler.java index 39d4c4e75a1a..0b3c923d1eb9 100644 --- a/gcloud-java-core/src/main/java/com/google/gcloud/ExceptionHandler.java +++ b/gcloud-java-core/src/main/java/com/google/gcloud/ExceptionHandler.java @@ -26,6 +26,7 @@ import java.io.Serializable; import java.lang.reflect.Method; +import java.util.Objects; import java.util.Set; import java.util.concurrent.Callable; @@ -259,6 +260,26 @@ boolean shouldRetry(Exception ex) { return retryResult == Interceptor.RetryResult.RETRY; } + @Override + public int hashCode() { + return Objects.hash(interceptors, retriableExceptions, nonRetriableExceptions, retryInfo); + } + + @Override + public boolean equals(Object obj) { + if (obj == this) { + return true; + } + if (!(obj instanceof ExceptionHandler)) { + return false; + } + ExceptionHandler other = (ExceptionHandler) obj; + return Objects.equals(interceptors, other.interceptors) + && Objects.equals(retriableExceptions, other.retriableExceptions) + && Objects.equals(nonRetriableExceptions, other.nonRetriableExceptions) + && Objects.equals(retryInfo, other.retryInfo); + } + /** * Returns an instance which retry any checked exception and abort on any runtime exception. */ diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/IamPolicy.java b/gcloud-java-core/src/main/java/com/google/gcloud/IamPolicy.java new file mode 100644 index 000000000000..9cce4b23c864 --- /dev/null +++ b/gcloud-java-core/src/main/java/com/google/gcloud/IamPolicy.java @@ -0,0 +1,231 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.ImmutableSet; + +import java.io.Serializable; +import java.util.Arrays; +import java.util.HashMap; +import java.util.HashSet; +import java.util.LinkedHashSet; +import java.util.Map; +import java.util.Objects; +import java.util.Set; + +/** + * Base class for Identity and Access Management (IAM) policies. IAM policies are used to specify + * access settings for Cloud Platform resources. A policy is a map of bindings. A binding assigns + * a set of identities to a role, where the identities can be user accounts, Google groups, Google + * domains, and service accounts. A role is a named list of permissions defined by IAM. + * + * @param the data type of roles (should be serializable) + * @see Policy + */ +public abstract class IamPolicy implements Serializable { + + private static final long serialVersionUID = 1114489978726897720L; + + private final Map> bindings; + private final String etag; + private final Integer version; + + /** + * Builder for an IAM Policy. + * + * @param the data type of roles + * @param the subclass extending this abstract builder + */ + public abstract static class Builder> { + + private final Map> bindings = new HashMap<>(); + private String etag; + private Integer version; + + /** + * Constructor for IAM Policy builder. + */ + protected Builder() {} + + /** + * Replaces the builder's map of bindings with the given map of bindings. + * + * @throws NullPointerException if the given map is null or contains any null keys or values + * @throws IllegalArgumentException if any identities in the given map are null + */ + public final B bindings(Map> bindings) { + checkNotNull(bindings, "The provided map of bindings cannot be null."); + for (Map.Entry> binding : bindings.entrySet()) { + checkNotNull(binding.getKey(), "The role cannot be null."); + Set identities = binding.getValue(); + checkNotNull(identities, "A role cannot be assigned to a null set of identities."); + checkArgument(!identities.contains(null), "Null identities are not permitted."); + } + this.bindings.clear(); + for (Map.Entry> binding : bindings.entrySet()) { + this.bindings.put(binding.getKey(), new HashSet(binding.getValue())); + } + return self(); + } + + /** + * Removes the role (and all identities associated with that role) from the policy. + */ + public final B removeRole(R role) { + bindings.remove(role); + return self(); + } + + /** + * Adds one or more identities to the policy under the role specified. + * + * @throws NullPointerException if the role or any of the identities is null. + */ + public final B addIdentity(R role, Identity first, Identity... others) { + String nullIdentityMessage = "Null identities are not permitted."; + checkNotNull(first, nullIdentityMessage); + checkNotNull(others, nullIdentityMessage); + for (Identity identity : others) { + checkNotNull(identity, nullIdentityMessage); + } + Set toAdd = new LinkedHashSet<>(); + toAdd.add(first); + toAdd.addAll(Arrays.asList(others)); + Set identities = bindings.get(checkNotNull(role, "The role cannot be null.")); + if (identities == null) { + identities = new HashSet(); + bindings.put(role, identities); + } + identities.addAll(toAdd); + return self(); + } + + /** + * Removes one or more identities from an existing binding. Does nothing if the binding + * associated with the provided role doesn't exist. + */ + public final B removeIdentity(R role, Identity first, Identity... others) { + Set identities = bindings.get(role); + if (identities != null) { + identities.remove(first); + identities.removeAll(Arrays.asList(others)); + } + if (identities != null && identities.isEmpty()) { + bindings.remove(role); + } + return self(); + } + + /** + * Sets the policy's etag. + * + *

Etags are used for optimistic concurrency control as a way to help prevent simultaneous + * updates of a policy from overwriting each other. It is strongly suggested that systems make + * use of the etag in the read-modify-write cycle to perform policy updates in order to avoid + * race conditions. An etag is returned in the response to getIamPolicy, and systems are + * expected to put that etag in the request to setIamPolicy to ensure that their change will be + * applied to the same version of the policy. If no etag is provided in the call to + * setIamPolicy, then the existing policy is overwritten blindly. + */ + protected final B etag(String etag) { + this.etag = etag; + return self(); + } + + /** + * Sets the version of the policy. The default version is 0, meaning only the "owner", "editor", + * and "viewer" roles are permitted. If the version is 1, you may also use other roles. + */ + protected final B version(Integer version) { + this.version = version; + return self(); + } + + @SuppressWarnings("unchecked") + private B self() { + return (B) this; + } + + public abstract IamPolicy build(); + } + + protected IamPolicy(Builder> builder) { + ImmutableMap.Builder> bindingsBuilder = ImmutableMap.builder(); + for (Map.Entry> binding : builder.bindings.entrySet()) { + bindingsBuilder.put(binding.getKey(), ImmutableSet.copyOf(binding.getValue())); + } + this.bindings = bindingsBuilder.build(); + this.etag = builder.etag; + this.version = builder.version; + } + + /** + * Returns a builder containing the properties of this IAM Policy. + */ + public abstract Builder> toBuilder(); + + /** + * The map of bindings that comprises the policy. + */ + public Map> bindings() { + return bindings; + } + + /** + * The policy's etag. + * + *

Etags are used for optimistic concurrency control as a way to help prevent simultaneous + * updates of a policy from overwriting each other. It is strongly suggested that systems make + * use of the etag in the read-modify-write cycle to perform policy updates in order to avoid + * race conditions. An etag is returned in the response to getIamPolicy, and systems are + * expected to put that etag in the request to setIamPolicy to ensure that their change will be + * applied to the same version of the policy. If no etag is provided in the call to + * setIamPolicy, then the existing policy is overwritten blindly. + */ + public String etag() { + return etag; + } + + /** + * Sets the version of the policy. The default version is 0, meaning only the "owner", "editor", + * and "viewer" roles are permitted. If the version is 1, you may also use other roles. + */ + public Integer version() { + return version; + } + + @Override + public final int hashCode() { + return Objects.hash(getClass(), bindings, etag, version); + } + + @Override + public final boolean equals(Object obj) { + if (obj == null || !getClass().equals(obj.getClass())) { + return false; + } + @SuppressWarnings("rawtypes") + IamPolicy other = (IamPolicy) obj; + return Objects.equals(bindings, other.bindings()) + && Objects.equals(etag, other.etag()) + && Objects.equals(version, other.version()); + } +} diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/Identity.java b/gcloud-java-core/src/main/java/com/google/gcloud/Identity.java new file mode 100644 index 000000000000..687a76ffc42c --- /dev/null +++ b/gcloud-java-core/src/main/java/com/google/gcloud/Identity.java @@ -0,0 +1,225 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud; + +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.common.base.CaseFormat; + +import java.io.Serializable; +import java.util.Objects; + +/** + * An identity in an {@link IamPolicy}. The following types of identities are permitted in IAM + * policies: + *

    + *
  • Google account + *
  • Service account + *
  • Google group + *
  • Google Apps domain + *
+ * + *

There are also two special identities that represent all users and all Google-authenticated + * accounts. + * + * @see Concepts + * related to identity + */ +public final class Identity implements Serializable { + + private static final long serialVersionUID = -8181841964597657446L; + + private final Type type; + private final String value; + + /** + * The types of IAM identities. + */ + public enum Type { + + /** + * Represents anyone who is on the internet; with or without a Google account. + */ + ALL_USERS, + + /** + * Represents anyone who is authenticated with a Google account or a service account. + */ + ALL_AUTHENTICATED_USERS, + + /** + * Represents a specific Google account. + */ + USER, + + /** + * Represents a service account. + */ + SERVICE_ACCOUNT, + + /** + * Represents a Google group. + */ + GROUP, + + /** + * Represents all the users of a Google Apps domain name. + */ + DOMAIN + } + + private Identity(Type type, String value) { + this.type = type; + this.value = value; + } + + public Type type() { + return type; + } + + /** + * Returns the string identifier for this identity. The value corresponds to: + *

    + *
  • email address (for identities of type {@code USER}, {@code SERVICE_ACCOUNT}, and + * {@code GROUP}) + *
  • domain (for identities of type {@code DOMAIN}) + *
  • {@code null} (for identities of type {@code ALL_USERS} and + * {@code ALL_AUTHENTICATED_USERS}) + *
+ */ + public String value() { + return value; + } + + /** + * Returns a new identity representing anyone who is on the internet; with or without a Google + * account. + */ + public static Identity allUsers() { + return new Identity(Type.ALL_USERS, null); + } + + /** + * Returns a new identity representing anyone who is authenticated with a Google account or a + * service account. + */ + public static Identity allAuthenticatedUsers() { + return new Identity(Type.ALL_AUTHENTICATED_USERS, null); + } + + /** + * Returns a new user identity. + * + * @param email An email address that represents a specific Google account. For example, + * alice@gmail.com or joe@example.com. + */ + public static Identity user(String email) { + return new Identity(Type.USER, checkNotNull(email)); + } + + /** + * Returns a new service account identity. + * + * @param email An email address that represents a service account. For example, + * my-other-app@appspot.gserviceaccount.com. + */ + public static Identity serviceAccount(String email) { + return new Identity(Type.SERVICE_ACCOUNT, checkNotNull(email)); + } + + /** + * Returns a new group identity. + * + * @param email An email address that represents a Google group. For example, + * admins@example.com. + */ + public static Identity group(String email) { + return new Identity(Type.GROUP, checkNotNull(email)); + } + + /** + * Returns a new domain identity. + * + * @param domain A Google Apps domain name that represents all the users of that domain. For + * example, google.com or example.com. + */ + public static Identity domain(String domain) { + return new Identity(Type.DOMAIN, checkNotNull(domain)); + } + + @Override + public int hashCode() { + return Objects.hash(value, type); + } + + @Override + public boolean equals(Object obj) { + if (!(obj instanceof Identity)) { + return false; + } + Identity other = (Identity) obj; + return Objects.equals(value, other.value()) && Objects.equals(type, other.type()); + } + + /** + * Returns the string value associated with the identity. Used primarily for converting from + * {@code Identity} objects to strings for protobuf-generated policies. + */ + public String strValue() { + switch (type) { + case ALL_USERS: + return "allUsers"; + case ALL_AUTHENTICATED_USERS: + return "allAuthenticatedUsers"; + case USER: + return "user:" + value; + case SERVICE_ACCOUNT: + return "serviceAccount:" + value; + case GROUP: + return "group:" + value; + case DOMAIN: + return "domain:" + value; + default: + throw new IllegalStateException("Unexpected identity type: " + type); + } + } + + /** + * Converts a string to an {@code Identity}. Used primarily for converting protobuf-generated + * policy identities to {@code Identity} objects. + */ + public static Identity valueOf(String identityStr) { + String[] info = identityStr.split(":"); + Type type = Type.valueOf(CaseFormat.LOWER_CAMEL.to(CaseFormat.UPPER_UNDERSCORE, info[0])); + switch (type) { + case ALL_USERS: + return Identity.allUsers(); + case ALL_AUTHENTICATED_USERS: + return Identity.allAuthenticatedUsers(); + case USER: + return Identity.user(info[1]); + case SERVICE_ACCOUNT: + return Identity.serviceAccount(info[1]); + case GROUP: + return Identity.group(info[1]); + case DOMAIN: + return Identity.domain(info[1]); + default: + throw new IllegalStateException("Unexpected identity type " + type); + } + } +} diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/Restorable.java b/gcloud-java-core/src/main/java/com/google/gcloud/Restorable.java index 90633c70046f..0b573522e370 100644 --- a/gcloud-java-core/src/main/java/com/google/gcloud/Restorable.java +++ b/gcloud-java-core/src/main/java/com/google/gcloud/Restorable.java @@ -21,14 +21,14 @@ * *

* A typical capture usage: - *

  {@code
+ * 
 {@code
  * X restorableObj; // X instanceof Restorable
  * RestorableState state = restorableObj.capture();
  * .. persist state
  * }
* * A typical restore usage: - *
  {@code
+ * 
 {@code
  * RestorableState state = ... // read from persistence
  * X restorableObj = state.restore();
  * ...
diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/ServiceOptions.java b/gcloud-java-core/src/main/java/com/google/gcloud/ServiceOptions.java
index 31e543809464..d45069434a26 100644
--- a/gcloud-java-core/src/main/java/com/google/gcloud/ServiceOptions.java
+++ b/gcloud-java-core/src/main/java/com/google/gcloud/ServiceOptions.java
@@ -523,9 +523,10 @@ public RetryParams retryParams() {
    * options.
    */
   public HttpRequestInitializer httpRequestInitializer() {
-    final HttpRequestInitializer delegate = authCredentials() != null
-        ? new HttpCredentialsAdapter(authCredentials().credentials().createScoped(scopes()))
-        : null;
+    final HttpRequestInitializer delegate =
+        authCredentials() != null && authCredentials.credentials() != null
+            ? new HttpCredentialsAdapter(authCredentials().credentials().createScoped(scopes()))
+            : null;
     return new HttpRequestInitializer() {
       @Override
       public void initialize(HttpRequest httpRequest) throws IOException {
diff --git a/gcloud-java-core/src/main/java/com/google/gcloud/package-info.java b/gcloud-java-core/src/main/java/com/google/gcloud/package-info.java
new file mode 100644
index 000000000000..d527640c99f9
--- /dev/null
+++ b/gcloud-java-core/src/main/java/com/google/gcloud/package-info.java
@@ -0,0 +1,20 @@
+/*
+ * Copyright 2016 Google Inc. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *       http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/**
+ * Core classes for the {@code gcloud-java} library.
+ */
+package com.google.gcloud;
diff --git a/gcloud-java-core/src/test/java/com/google/gcloud/BaseSerializationTest.java b/gcloud-java-core/src/test/java/com/google/gcloud/BaseSerializationTest.java
new file mode 100644
index 000000000000..e9ab3d47984b
--- /dev/null
+++ b/gcloud-java-core/src/test/java/com/google/gcloud/BaseSerializationTest.java
@@ -0,0 +1,86 @@
+/*
+ * Copyright 2016 Google Inc. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *       http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.google.gcloud;
+
+import static com.google.common.base.MoreObjects.firstNonNull;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotSame;
+
+import org.junit.Test;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.ObjectInputStream;
+import java.io.ObjectOutputStream;
+import java.io.Serializable;
+
+/**
+ * Base class for serialization tests. To use this class in your tests override the
+ * {@code serializableObjects()} method to return all objects that must be serializable. Also
+ * override {@code restorableObjects()} method to return all restorable objects whose state must be
+ * tested for proper serialization. Both methods can return {@code null} if no such object needs to
+ * be tested.
+ */
+public abstract class BaseSerializationTest {
+
+  /**
+   * Returns all objects for which correct serialization must be tested.
+   */
+  protected abstract Serializable[] serializableObjects();
+
+  /**
+   * Returns all restorable objects whose state must be tested for proper serialization.
+   */
+  protected abstract Restorable[] restorableObjects();
+
+  @Test
+  public void testSerializableObjects() throws Exception {
+    for (Serializable obj : firstNonNull(serializableObjects(), new Serializable[0])) {
+      Object copy = serializeAndDeserialize(obj);
+      assertEquals(obj, obj);
+      assertEquals(obj, copy);
+      assertEquals(obj.hashCode(), copy.hashCode());
+      assertEquals(obj.toString(), copy.toString());
+      assertNotSame(obj, copy);
+      assertEquals(copy, copy);
+    }
+  }
+
+  @Test
+  public void testRestorableObjects() throws Exception {
+    for (Restorable restorable : firstNonNull(restorableObjects(), new Restorable[0])) {
+      RestorableState state = restorable.capture();
+      RestorableState deserializedState = serializeAndDeserialize(state);
+      assertEquals(state, deserializedState);
+      assertEquals(state.hashCode(), deserializedState.hashCode());
+      assertEquals(state.toString(), deserializedState.toString());
+    }
+  }
+
+  @SuppressWarnings("unchecked")
+  public  T serializeAndDeserialize(T obj) throws IOException, ClassNotFoundException {
+    ByteArrayOutputStream bytes = new ByteArrayOutputStream();
+    try (ObjectOutputStream output = new ObjectOutputStream(bytes)) {
+      output.writeObject(obj);
+    }
+    try (ObjectInputStream input =
+        new ObjectInputStream(new ByteArrayInputStream(bytes.toByteArray()))) {
+      return (T) input.readObject();
+    }
+  }
+}
diff --git a/gcloud-java-core/src/test/java/com/google/gcloud/BaseWriteChannelTest.java b/gcloud-java-core/src/test/java/com/google/gcloud/BaseWriteChannelTest.java
index e49a17b019e0..6d5306a3bc7f 100644
--- a/gcloud-java-core/src/test/java/com/google/gcloud/BaseWriteChannelTest.java
+++ b/gcloud-java-core/src/test/java/com/google/gcloud/BaseWriteChannelTest.java
@@ -32,6 +32,7 @@
 import java.io.IOException;
 import java.io.Serializable;
 import java.nio.ByteBuffer;
+import java.nio.channels.ClosedChannelException;
 import java.util.Arrays;
 import java.util.Random;
 
@@ -102,8 +103,7 @@ public void testClose() throws IOException {
   @Test
   public void testValidateOpen() throws IOException {
     channel.close();
-    thrown.expect(IOException.class);
-    thrown.expectMessage("stream is closed");
+    thrown.expect(ClosedChannelException.class);
     channel.write(ByteBuffer.allocate(42));
   }
 
diff --git a/gcloud-java-core/src/test/java/com/google/gcloud/IamPolicyTest.java b/gcloud-java-core/src/test/java/com/google/gcloud/IamPolicyTest.java
new file mode 100644
index 000000000000..235c2c2b1c85
--- /dev/null
+++ b/gcloud-java-core/src/test/java/com/google/gcloud/IamPolicyTest.java
@@ -0,0 +1,221 @@
+/*
+ * Copyright 2016 Google Inc. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *       http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.google.gcloud;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+
+import org.junit.Test;
+
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+public class IamPolicyTest {
+
+  private static final Identity ALL_USERS = Identity.allUsers();
+  private static final Identity ALL_AUTH_USERS = Identity.allAuthenticatedUsers();
+  private static final Identity USER = Identity.user("abc@gmail.com");
+  private static final Identity SERVICE_ACCOUNT =
+      Identity.serviceAccount("service-account@gmail.com");
+  private static final Identity GROUP = Identity.group("group@gmail.com");
+  private static final Identity DOMAIN = Identity.domain("google.com");
+  private static final Map> BINDINGS = ImmutableMap.of(
+      "viewer",
+      ImmutableSet.of(USER, SERVICE_ACCOUNT, ALL_USERS),
+      "editor",
+      ImmutableSet.of(ALL_AUTH_USERS, GROUP, DOMAIN));
+  private static final PolicyImpl SIMPLE_POLICY = PolicyImpl.builder()
+      .addIdentity("viewer", USER, SERVICE_ACCOUNT, ALL_USERS)
+      .addIdentity("editor", ALL_AUTH_USERS, GROUP, DOMAIN)
+      .build();
+  private static final PolicyImpl FULL_POLICY =
+      new PolicyImpl.Builder(SIMPLE_POLICY.bindings(), "etag", 1).build();
+
+  static class PolicyImpl extends IamPolicy {
+
+    static class Builder extends IamPolicy.Builder {
+
+      private Builder() {}
+
+      private Builder(Map> bindings, String etag, Integer version) {
+        bindings(bindings).etag(etag).version(version);
+      }
+
+      @Override
+      public PolicyImpl build() {
+        return new PolicyImpl(this);
+      }
+    }
+
+    PolicyImpl(Builder builder) {
+      super(builder);
+    }
+
+    @Override
+    public Builder toBuilder() {
+      return new Builder(bindings(), etag(), version());
+    }
+
+    static Builder builder() {
+      return new Builder();
+    }
+  }
+
+  @Test
+  public void testBuilder() {
+    assertEquals(BINDINGS, FULL_POLICY.bindings());
+    assertEquals("etag", FULL_POLICY.etag());
+    assertEquals(1, FULL_POLICY.version().intValue());
+    Map> editorBinding =
+        ImmutableMap.>builder().put("editor", BINDINGS.get("editor")).build();
+    PolicyImpl policy = FULL_POLICY.toBuilder().bindings(editorBinding).build();
+    assertEquals(editorBinding, policy.bindings());
+    assertEquals("etag", policy.etag());
+    assertEquals(1, policy.version().intValue());
+    policy = SIMPLE_POLICY.toBuilder().removeRole("editor").build();
+    assertEquals(ImmutableMap.of("viewer", BINDINGS.get("viewer")), policy.bindings());
+    assertNull(policy.etag());
+    assertNull(policy.version());
+    policy = policy.toBuilder()
+        .removeIdentity("viewer", USER, ALL_USERS)
+        .addIdentity("viewer", DOMAIN, GROUP)
+        .build();
+    assertEquals(ImmutableMap.of("viewer", ImmutableSet.of(SERVICE_ACCOUNT, DOMAIN, GROUP)),
+        policy.bindings());
+    assertNull(policy.etag());
+    assertNull(policy.version());
+    policy = PolicyImpl.builder()
+        .removeIdentity("viewer", USER)
+        .addIdentity("owner", USER, SERVICE_ACCOUNT)
+        .addIdentity("editor", GROUP)
+        .removeIdentity("editor", GROUP)
+        .build();
+    assertEquals(
+        ImmutableMap.of("owner", ImmutableSet.of(USER, SERVICE_ACCOUNT)), policy.bindings());
+    assertNull(policy.etag());
+    assertNull(policy.version());
+  }
+
+  @Test
+  public void testIllegalPolicies() {
+    try {
+      PolicyImpl.builder().addIdentity(null, USER);
+      fail("Null role should cause exception.");
+    } catch (NullPointerException ex) {
+      assertEquals("The role cannot be null.", ex.getMessage());
+    }
+    try {
+      PolicyImpl.builder().addIdentity("viewer", null, USER);
+      fail("Null identity should cause exception.");
+    } catch (NullPointerException ex) {
+      assertEquals("Null identities are not permitted.", ex.getMessage());
+    }
+    try {
+      PolicyImpl.builder().addIdentity("viewer", USER, (Identity[]) null);
+      fail("Null identity should cause exception.");
+    } catch (NullPointerException ex) {
+      assertEquals("Null identities are not permitted.", ex.getMessage());
+    }
+    try {
+      PolicyImpl.builder().bindings(null);
+      fail("Null bindings map should cause exception.");
+    } catch (NullPointerException ex) {
+      assertEquals("The provided map of bindings cannot be null.", ex.getMessage());
+    }
+    try {
+      Map> bindings = new HashMap<>();
+      bindings.put("viewer", null);
+      PolicyImpl.builder().bindings(bindings);
+      fail("Null set of identities should cause exception.");
+    } catch (NullPointerException ex) {
+      assertEquals("A role cannot be assigned to a null set of identities.", ex.getMessage());
+    }
+    try {
+      Map> bindings = new HashMap<>();
+      Set identities = new HashSet<>();
+      identities.add(null);
+      bindings.put("viewer", identities);
+      PolicyImpl.builder().bindings(bindings);
+      fail("Null identity should cause exception.");
+    } catch (IllegalArgumentException ex) {
+      assertEquals("Null identities are not permitted.", ex.getMessage());
+    }
+  }
+
+  @Test
+  public void testEqualsHashCode() {
+    assertNotNull(FULL_POLICY);
+    PolicyImpl emptyPolicy = PolicyImpl.builder().build();
+    AnotherPolicyImpl anotherPolicy = new AnotherPolicyImpl.Builder().build();
+    assertNotEquals(emptyPolicy, anotherPolicy);
+    assertNotEquals(emptyPolicy.hashCode(), anotherPolicy.hashCode());
+    assertNotEquals(FULL_POLICY, SIMPLE_POLICY);
+    assertNotEquals(FULL_POLICY.hashCode(), SIMPLE_POLICY.hashCode());
+    PolicyImpl copy = SIMPLE_POLICY.toBuilder().build();
+    assertEquals(SIMPLE_POLICY, copy);
+    assertEquals(SIMPLE_POLICY.hashCode(), copy.hashCode());
+  }
+
+  @Test
+  public void testBindings() {
+    assertTrue(PolicyImpl.builder().build().bindings().isEmpty());
+    assertEquals(BINDINGS, SIMPLE_POLICY.bindings());
+  }
+
+  @Test
+  public void testEtag() {
+    assertNull(SIMPLE_POLICY.etag());
+    assertEquals("etag", FULL_POLICY.etag());
+  }
+
+  @Test
+  public void testVersion() {
+    assertNull(SIMPLE_POLICY.version());
+    assertEquals(1, FULL_POLICY.version().intValue());
+  }
+
+  static class AnotherPolicyImpl extends IamPolicy {
+
+    static class Builder extends IamPolicy.Builder {
+
+      private Builder() {}
+
+      @Override
+      public AnotherPolicyImpl build() {
+        return new AnotherPolicyImpl(this);
+      }
+    }
+
+    AnotherPolicyImpl(Builder builder) {
+      super(builder);
+    }
+
+    @Override
+    public Builder toBuilder() {
+      return new Builder();
+    }
+  }
+}
diff --git a/gcloud-java-core/src/test/java/com/google/gcloud/IdentityTest.java b/gcloud-java-core/src/test/java/com/google/gcloud/IdentityTest.java
new file mode 100644
index 000000000000..a42bc9db7abd
--- /dev/null
+++ b/gcloud-java-core/src/test/java/com/google/gcloud/IdentityTest.java
@@ -0,0 +1,105 @@
+/*
+ * Copyright 2016 Google Inc. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *       http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.google.gcloud;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNull;
+
+import org.junit.Test;
+
+public class IdentityTest {
+
+  private static final Identity ALL_USERS = Identity.allUsers();
+  private static final Identity ALL_AUTH_USERS = Identity.allAuthenticatedUsers();
+  private static final Identity USER = Identity.user("abc@gmail.com");
+  private static final Identity SERVICE_ACCOUNT =
+      Identity.serviceAccount("service-account@gmail.com");
+  private static final Identity GROUP = Identity.group("group@gmail.com");
+  private static final Identity DOMAIN = Identity.domain("google.com");
+
+  @Test
+  public void testAllUsers() {
+    assertEquals(Identity.Type.ALL_USERS, ALL_USERS.type());
+    assertNull(ALL_USERS.value());
+  }
+
+  @Test
+  public void testAllAuthenticatedUsers() {
+    assertEquals(Identity.Type.ALL_AUTHENTICATED_USERS, ALL_AUTH_USERS.type());
+    assertNull(ALL_AUTH_USERS.value());
+  }
+
+  @Test
+  public void testUser() {
+    assertEquals(Identity.Type.USER, USER.type());
+    assertEquals("abc@gmail.com", USER.value());
+  }
+
+  @Test(expected = NullPointerException.class)
+  public void testUserNullEmail() {
+    Identity.user(null);
+  }
+
+  @Test
+  public void testServiceAccount() {
+    assertEquals(Identity.Type.SERVICE_ACCOUNT, SERVICE_ACCOUNT.type());
+    assertEquals("service-account@gmail.com", SERVICE_ACCOUNT.value());
+  }
+
+  @Test(expected = NullPointerException.class)
+  public void testServiceAccountNullEmail() {
+    Identity.serviceAccount(null);
+  }
+
+  @Test
+  public void testGroup() {
+    assertEquals(Identity.Type.GROUP, GROUP.type());
+    assertEquals("group@gmail.com", GROUP.value());
+  }
+
+  @Test(expected = NullPointerException.class)
+  public void testGroupNullEmail() {
+    Identity.group(null);
+  }
+
+  @Test
+  public void testDomain() {
+    assertEquals(Identity.Type.DOMAIN, DOMAIN.type());
+    assertEquals("google.com", DOMAIN.value());
+  }
+
+  @Test(expected = NullPointerException.class)
+  public void testDomainNullId() {
+    Identity.domain(null);
+  }
+
+  @Test
+  public void testIdentityToAndFromPb() {
+    compareIdentities(ALL_USERS, Identity.valueOf(ALL_USERS.strValue()));
+    compareIdentities(ALL_AUTH_USERS, Identity.valueOf(ALL_AUTH_USERS.strValue()));
+    compareIdentities(USER, Identity.valueOf(USER.strValue()));
+    compareIdentities(SERVICE_ACCOUNT, Identity.valueOf(SERVICE_ACCOUNT.strValue()));
+    compareIdentities(GROUP, Identity.valueOf(GROUP.strValue()));
+    compareIdentities(DOMAIN, Identity.valueOf(DOMAIN.strValue()));
+  }
+
+  private void compareIdentities(Identity expected, Identity actual) {
+    assertEquals(expected, actual);
+    assertEquals(expected.type(), actual.type());
+    assertEquals(expected.value(), actual.value());
+  }
+}
diff --git a/gcloud-java-core/src/test/java/com/google/gcloud/SerializationTest.java b/gcloud-java-core/src/test/java/com/google/gcloud/SerializationTest.java
new file mode 100644
index 000000000000..3255a17333aa
--- /dev/null
+++ b/gcloud-java-core/src/test/java/com/google/gcloud/SerializationTest.java
@@ -0,0 +1,100 @@
+/*
+ * Copyright 2016 Google Inc. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *       http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.google.gcloud;
+
+import com.google.common.collect.ImmutableList;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.Serializable;
+
+public class SerializationTest extends BaseSerializationTest {
+
+  private static class SomeIamPolicy extends IamPolicy {
+
+    private static final long serialVersionUID = 271243551016958285L;
+
+    private static class Builder extends IamPolicy.Builder {
+
+      @Override
+      public SomeIamPolicy build() {
+        return new SomeIamPolicy(this);
+      }
+    }
+
+    protected SomeIamPolicy(Builder builder) {
+      super(builder);
+    }
+
+    @Override
+    public Builder toBuilder() {
+      return new Builder();
+    }
+  }
+
+  private static final BaseServiceException BASE_SERVICE_EXCEPTION =
+      new BaseServiceException(42, "message", "reason", true);
+  private static final ExceptionHandler EXCEPTION_HANDLER = ExceptionHandler.defaultInstance();
+  private static final Identity IDENTITY = Identity.allAuthenticatedUsers();
+  private static final PageImpl PAGE =
+      new PageImpl<>(null, "cursor", ImmutableList.of("string1", "string2"));
+  private static final RetryParams RETRY_PARAMS = RetryParams.defaultInstance();
+  private static final SomeIamPolicy SOME_IAM_POLICY = new SomeIamPolicy.Builder().build();
+  private static final String JSON_KEY = "{\n"
+      + "  \"private_key_id\": \"somekeyid\",\n"
+      + "  \"private_key\": \"-----BEGIN PRIVATE KEY-----\\nMIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggS"
+      + "kAgEAAoIBAQC+K2hSuFpAdrJI\\nnCgcDz2M7t7bjdlsadsasad+fvRSW6TjNQZ3p5LLQY1kSZRqBqylRkzteMOyHg"
+      + "aR\\n0Pmxh3ILCND5men43j3h4eDbrhQBuxfEMalkG92sL+PNQSETY2tnvXryOvmBRwa/\\nQP/9dJfIkIDJ9Fw9N4"
+      + "Bhhhp6mCcRpdQjV38H7JsyJ7lih/oNjECgYAt\\nknddadwkwewcVxHFhcZJO+XWf6ofLUXpRwiTZakGMn8EE1uVa2"
+      + "LgczOjwWHGi99MFjxSer5m9\\n1tCa3/KEGKiS/YL71JvjwX3mb+cewlkcmweBKZHM2JPTk0ZednFSpVZMtycjkbLa"
+      + "\\ndYOS8V85AgMBewECggEBAKksaldajfDZDV6nGqbFjMiizAKJolr/M3OQw16K6o3/\\n0S31xIe3sSlgW0+UbYlF"
+      + "4U8KifhManD1apVSC3csafaspP4RZUHFhtBywLO9pR5c\\nr6S5aLp+gPWFyIp1pfXbWGvc5VY/v9x7ya1VEa6rXvL"
+      + "sKupSeWAW4tMj3eo/64ge\\nsdaceaLYw52KeBYiT6+vpsnYrEkAHO1fF/LavbLLOFJmFTMxmsNaG0tuiJHgjshB\\"
+      + "n82DpMCbXG9YcCgI/DbzuIjsdj2JC1cascSP//3PmefWysucBQe7Jryb6NQtASmnv\\nCdDw/0jmZTEjpe4S1lxfHp"
+      + "lAhHFtdgYTvyYtaLZiVVkCgYEA8eVpof2rceecw/I6\\n5ng1q3Hl2usdWV/4mZMvR0fOemacLLfocX6IYxT1zA1FF"
+      + "JlbXSRsJMf/Qq39mOR2\\nSpW+hr4jCoHeRVYLgsbggtrevGmILAlNoqCMpGZ6vDmJpq6ECV9olliDvpPgWOP+\\nm"
+      + "YPDreFBGxWvQrADNbRt2dmGsrsCgYEAyUHqB2wvJHFqdmeBsaacewzV8x9WgmeX\\ngUIi9REwXlGDW0Mz50dxpxcK"
+      + "CAYn65+7TCnY5O/jmL0VRxU1J2mSWyWTo1C+17L0\\n3fUqjxL1pkefwecxwecvC+gFFYdJ4CQ/MHHXU81Lwl1iWdF"
+      + "Cd2UoGddYaOF+KNeM\\nHC7cmqra+JsCgYEAlUNywzq8nUg7282E+uICfCB0LfwejuymR93CtsFgb7cRd6ak\\nECR"
+      + "8FGfCpH8ruWJINllbQfcHVCX47ndLZwqv3oVFKh6pAS/vVI4dpOepP8++7y1u\\ncoOvtreXCX6XqfrWDtKIvv0vjl"
+      + "HBhhhp6mCcRpdQjV38H7JsyJ7lih/oNjECgYAt\\nkndj5uNl5SiuVxHFhcZJO+XWf6ofLUregtevZakGMn8EE1uVa"
+      + "2AY7eafmoU/nZPT\\n00YB0TBATdCbn/nBSuKDESkhSg9s2GEKQZG5hBmL5uCMfo09z3SfxZIhJdlerreP\\nJ7gSi"
+      + "dI12N+EZxYd4xIJh/HFDgp7RRO87f+WJkofMQKBgGTnClK1VMaCRbJZPriw\\nEfeFCoOX75MxKwXs6xgrw4W//AYG"
+      + "GUjDt83lD6AZP6tws7gJ2IwY/qP7+lyhjEqN\\nHtfPZRGFkGZsdaksdlaksd323423d+15/UvrlRSFPNj1tWQmNKk"
+      + "XyRDW4IG1Oa2p\\nrALStNBx5Y9t0/LQnFI4w3aG\\n-----END PRIVATE KEY-----\\n\",\n"
+      + "  \"client_email\": \"someclientid@developer.gserviceaccount.com\",\n"
+      + "  \"client_id\": \"someclientid.apps.googleusercontent.com\",\n"
+      + "  \"type\": \"service_account\"\n"
+      + "}";
+
+  @Override
+  protected Serializable[] serializableObjects() {
+    return new Serializable[]{BASE_SERVICE_EXCEPTION, EXCEPTION_HANDLER, IDENTITY, PAGE,
+        RETRY_PARAMS, SOME_IAM_POLICY};
+  }
+
+  @Override
+  protected Restorable[] restorableObjects() {
+    try {
+      return new Restorable[]{AuthCredentials.createForAppEngine(), AuthCredentials.noAuth(),
+          AuthCredentials.createForJson(new ByteArrayInputStream(JSON_KEY.getBytes()))};
+    } catch (IOException ex) {
+      // never reached
+      throw new RuntimeException(ex);
+    }
+  }
+}
diff --git a/gcloud-java-datastore/README.md b/gcloud-java-datastore/README.md
index 8c0ae1950e77..0d89a0a07e3e 100644
--- a/gcloud-java-datastore/README.md
+++ b/gcloud-java-datastore/README.md
@@ -7,6 +7,7 @@ Java idiomatic client for [Google Cloud Datastore] (https://cloud.google.com/dat
 [![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master)
 [![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-datastore.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-datastore.svg)
 [![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java)
+[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969)
 
 -  [Homepage] (https://googlecloudplatform.github.io/gcloud-java/)
 -  [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/datastore/package-summary.html)
@@ -21,21 +22,21 @@ If you are using Maven, add this to your pom.xml file
 
   com.google.gcloud
   gcloud-java-datastore
-  0.1.3
+  0.1.5
 
 ```
 If you are using Gradle, add this to your dependencies
 ```Groovy
-compile 'com.google.gcloud:gcloud-java-datastore:0.1.3'
+compile 'com.google.gcloud:gcloud-java-datastore:0.1.5'
 ```
 If you are using SBT, add this to your dependencies
 ```Scala
-libraryDependencies += "com.google.gcloud" % "gcloud-java-datastore" % "0.1.3"
+libraryDependencies += "com.google.gcloud" % "gcloud-java-datastore" % "0.1.5"
 ```
 
 Example Application
 --------------------
-[`DatastoreExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/datastore/DatastoreExample.java) is a simple command line interface for the Cloud Datastore.  Read more about using the application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/DatastoreExample.html).
+[`DatastoreExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/datastore/DatastoreExample.java) is a simple command line interface for the Cloud Datastore.  Read more about using the application on the [`DatastoreExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/datastore/DatastoreExample.html).
 
 Authentication
 --------------
diff --git a/gcloud-java-datastore/pom.xml b/gcloud-java-datastore/pom.xml
index a76e4f9d9cbd..f3b46e22b3c8 100644
--- a/gcloud-java-datastore/pom.xml
+++ b/gcloud-java-datastore/pom.xml
@@ -10,7 +10,7 @@
   
     com.google.gcloud
     gcloud-java-pom
-    0.1.4-SNAPSHOT
+    0.1.6-SNAPSHOT
   
   
     gcloud-java-datastore
@@ -24,7 +24,7 @@
     
       com.google.apis
       google-api-services-datastore-protobuf
-      v1beta2-rev1-2.1.2
+      v1beta2-rev1-4.0.0
       compile
       
         
@@ -33,6 +33,13 @@
         
       
     
+    
+      ${project.groupId}
+      gcloud-java-core
+      ${project.version}
+      test-jar
+      test
+    
     
       junit
       junit
@@ -42,7 +49,7 @@
     
       org.easymock
       easymock
-      3.3
+      3.4
       test
     
   
diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/BaseEntity.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/BaseEntity.java
index d674a5e242ad..20c0b13e5001 100644
--- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/BaseEntity.java
+++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/BaseEntity.java
@@ -33,6 +33,7 @@
 import com.google.protobuf.InvalidProtocolBufferException;
 
 import java.util.HashMap;
+import java.util.LinkedList;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
@@ -128,61 +129,286 @@ public B remove(String name) {
       return self();
     }
 
+    /**
+     * Sets a property.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, Value value) {
       properties.put(name, value);
       return self();
     }
 
+    /**
+     * Sets a property of type {@link StringValue}.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, String value) {
       properties.put(name, of(value));
       return self();
     }
 
+    /**
+     * Sets a list property containing elements of type {@link StringValue}.
+     *
+     * @param name name of the property
+     * @param first the first string in the list
+     * @param second the second string in the list
+     * @param others other strings in the list
+     */
+    public B set(String name, String first, String second, String... others) {
+      List values = new LinkedList<>();
+      values.add(of(first));
+      values.add(of(second));
+      for (String other : others) {
+        values.add(of(other));
+      }
+      properties.put(name, of(values));
+      return self();
+    }
+
+    /**
+     * Sets a property of type {@link LongValue}.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, long value) {
       properties.put(name, of(value));
       return self();
     }
 
+    /**
+     * Sets a list property containing elements of type {@link LongValue}.
+     *
+     * @param name name of the property
+     * @param first the first long in the list
+     * @param second the second long in the list
+     * @param others other longs in the list
+     */
+    public B set(String name, long first, long second, long... others) {
+      List values = new LinkedList<>();
+      values.add(of(first));
+      values.add(of(second));
+      for (long other : others) {
+        values.add(of(other));
+      }
+      properties.put(name, of(values));
+      return self();
+    }
+
+    /**
+     * Sets a property of type {@link DoubleValue}.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, double value) {
       properties.put(name, of(value));
       return self();
     }
 
+    /**
+     * Sets a list property containing elements of type {@link DoubleValue}.
+     *
+     * @param name name of the property
+     * @param first the first double in the list
+     * @param second the second double in the list
+     * @param others other doubles in the list
+     */
+    public B set(String name, double first, double second, double... others) {
+      List values = new LinkedList<>();
+      values.add(of(first));
+      values.add(of(second));
+      for (double other : others) {
+        values.add(of(other));
+      }
+      properties.put(name, of(values));
+      return self();
+    }
+
+    /**
+     * Sets a property of type {@link BooleanValue}.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, boolean value) {
       properties.put(name, of(value));
       return self();
     }
 
+    /**
+     * Sets a list property containing elements of type {@link BooleanValue}.
+     *
+     * @param name name of the property
+     * @param first the first boolean in the list
+     * @param second the second boolean in the list
+     * @param others other booleans in the list
+     */
+    public B set(String name, boolean first, boolean second, boolean... others) {
+      List values = new LinkedList<>();
+      values.add(of(first));
+      values.add(of(second));
+      for (boolean other : others) {
+        values.add(of(other));
+      }
+      properties.put(name, of(values));
+      return self();
+    }
+
+    /**
+     * Sets a property of type {@link DateTimeValue}.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, DateTime value) {
       properties.put(name, of(value));
       return self();
     }
 
+    /**
+     * Sets a list property containing elements of type {@link DateTimeValue}.
+     *
+     * @param name name of the property
+     * @param first the first {@link DateTime} in the list
+     * @param second the second {@link DateTime} in the list
+     * @param others other {@link DateTime}s in the list
+     */
+    public B set(String name, DateTime first, DateTime second, DateTime... others) {
+      List values = new LinkedList<>();
+      values.add(of(first));
+      values.add(of(second));
+      for (DateTime other : others) {
+        values.add(of(other));
+      }
+      properties.put(name, of(values));
+      return self();
+    }
+
+    /**
+     * Sets a property of type {@link KeyValue}.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, Key value) {
       properties.put(name, of(value));
       return self();
     }
 
+    /**
+     * Sets a list property containing elements of type {@link KeyValue}.
+     *
+     * @param name name of the property
+     * @param first the first {@link Key} in the list
+     * @param second the second {@link Key} in the list
+     * @param others other {@link Key}s in the list
+     */
+    public B set(String name, Key first, Key second, Key... others) {
+      List values = new LinkedList<>();
+      values.add(of(first));
+      values.add(of(second));
+      for (Key other : others) {
+        values.add(of(other));
+      }
+      properties.put(name, of(values));
+      return self();
+    }
+
+    /**
+     * Sets a property of type {@link EntityValue}.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, FullEntity value) {
       properties.put(name, of(value));
       return self();
     }
 
+    /**
+     * Sets a list property containing elements of type {@link EntityValue}.
+     *
+     * @param name name of the property
+     * @param first the first {@link FullEntity} in the list
+     * @param second the second {@link FullEntity} in the list
+     * @param others other entities in the list
+     */
+    public B set(String name, FullEntity first, FullEntity second, FullEntity... others) {
+      List values = new LinkedList<>();
+      values.add(of(first));
+      values.add(of(second));
+      for (FullEntity other : others) {
+        values.add(of(other));
+      }
+      properties.put(name, of(values));
+      return self();
+    }
+
+    /**
+     * Sets a property of type {@link ListValue}.
+     *
+     * @param name name of the property
+     * @param values list of values associated with the property
+     */
     public B set(String name, List> values) {
       properties.put(name, of(values));
       return self();
     }
 
-    public B set(String name, Value value, Value... other) {
-      properties.put(name, of(value, other));
+    /**
+     * Sets a property of type {@link ListValue}.
+     *
+     * @param name name of the property
+     * @param first the first value in the list
+     * @param second the second value in the list
+     * @param others other values in the list
+     */
+    public B set(String name, Value first, Value second, Value... others) {
+      properties.put(name, ListValue.builder().addValue(first).addValue(second, others).build());
       return self();
     }
 
+    /**
+     * Sets a property of type {@link BlobValue}.
+     *
+     * @param name name of the property
+     * @param value value associated with the property
+     */
     public B set(String name, Blob value) {
       properties.put(name, of(value));
       return self();
     }
 
+    /**
+     * Sets a list property containing elements of type {@link BlobValue}.
+     *
+     * @param name name of the property
+     * @param first the first {@link Blob} in the list
+     * @param second the second {@link Blob} in the list
+     * @param others other {@link Blob}s in the list
+     */
+    public B set(String name, Blob first, Blob second, Blob... others) {
+      List values = new LinkedList<>();
+      values.add(of(first));
+      values.add(of(second));
+      for (Blob other : others) {
+        values.add(of(other));
+      }
+      properties.put(name, of(values));
+      return self();
+    }
+
+    /**
+     * Sets a property of type {@code NullValue}.
+     *
+     * @param name name of the property
+     */
     public B setNull(String name) {
       properties.put(name, of());
       return self();
@@ -348,8 +574,8 @@ public  FullEntity getEntity(String name) {
    * @throws ClassCastException if value is not a list of values
    */
   @SuppressWarnings("unchecked")
-  public List> getList(String name) {
-    return ((Value>>) getValue(name)).get();
+  public > List getList(String name) {
+    return (List) getValue(name).get();
   }
 
   /**
diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/BaseKey.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/BaseKey.java
index a8ad7d4e7734..4ab6f51b6767 100644
--- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/BaseKey.java
+++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/BaseKey.java
@@ -157,6 +157,8 @@ public String kind() {
     return leaf().kind();
   }
 
+  abstract BaseKey parent();
+
   @Override
   public int hashCode() {
     return Objects.hash(projectId(), namespace(), path());
diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/Batch.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/Batch.java
index 75a5d1381403..5306a685195a 100644
--- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/Batch.java
+++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/Batch.java
@@ -24,14 +24,14 @@
  * to the Datastore upon {@link #submit}.
  * A usage example:
  * 
 {@code
- *   Entity entity1 = datastore.get(key1);
- *   Batch batch = datastore.newBatch();
- *   Entity entity2 = Entity.builder(key2).set("name", "John").build();
- *   entity1 = Entity.builder(entity1).clear().setNull("bla").build();
- *   Entity entity3 = Entity.builder(key3).set("title", "title").build();
- *   batch.update(entity1);
- *   batch.add(entity2, entity3);
- *   batch.submit();
+ * Entity entity1 = datastore.get(key1);
+ * Batch batch = datastore.newBatch();
+ * Entity entity2 = Entity.builder(key2).set("name", "John").build();
+ * entity1 = Entity.builder(entity1).clear().setNull("bla").build();
+ * Entity entity3 = Entity.builder(key3).set("title", "title").build();
+ * batch.update(entity1);
+ * batch.add(entity2, entity3);
+ * batch.submit();
  * } 
*/ public interface Batch extends DatastoreBatchWriter { diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/DatastoreImpl.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/DatastoreImpl.java index 92d18ed4787c..49a5728a4da9 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/DatastoreImpl.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/DatastoreImpl.java @@ -26,7 +26,7 @@ import com.google.gcloud.RetryHelper; import com.google.gcloud.RetryHelper.RetryHelperException; import com.google.gcloud.RetryParams; -import com.google.gcloud.spi.DatastoreRpc; +import com.google.gcloud.datastore.spi.DatastoreRpc; import com.google.protobuf.ByteString; import java.util.Arrays; diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/DatastoreOptions.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/DatastoreOptions.java index 2ec0f2be8f2b..db1a5f800ce8 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/DatastoreOptions.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/DatastoreOptions.java @@ -24,9 +24,9 @@ import com.google.common.collect.ImmutableSet; import com.google.common.collect.Iterables; import com.google.gcloud.ServiceOptions; -import com.google.gcloud.spi.DatastoreRpc; -import com.google.gcloud.spi.DatastoreRpcFactory; -import com.google.gcloud.spi.DefaultDatastoreRpc; +import com.google.gcloud.datastore.spi.DatastoreRpc; +import com.google.gcloud.datastore.spi.DatastoreRpcFactory; +import com.google.gcloud.datastore.spi.DefaultDatastoreRpc; import java.lang.reflect.Method; import java.util.Iterator; diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/GqlQuery.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/GqlQuery.java index e6ae166dbf07..7c03b69d9f39 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/GqlQuery.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/GqlQuery.java @@ -43,27 +43,27 @@ *

A usage example:

* *

When the type of the results is known the preferred usage would be: - *

{@code
- *   Query query =
- *       Query.gqlQueryBuilder(Query.ResultType.ENTITY, "select * from kind").build();
- *   QueryResults results = datastore.run(query);
- *   while (results.hasNext()) {
- *     Entity entity = results.next();
- *     ...
- *   }
+ * 
 {@code
+ * Query query =
+ *     Query.gqlQueryBuilder(Query.ResultType.ENTITY, "select * from kind").build();
+ * QueryResults results = datastore.run(query);
+ * while (results.hasNext()) {
+ *   Entity entity = results.next();
+ *   ...
+ * }
  * } 
* *

When the type of the results is unknown you can use this approach: - *

{@code
- *   Query query = Query.gqlQueryBuilder("select __key__ from kind").build();
- *   QueryResults results = datastore.run(query);
- *   if (Key.class.isAssignableFrom(results.resultClass())) {
- *     QueryResults keys = (QueryResults) results;
- *     while (keys.hasNext()) {
- *       Key key = keys.next();
- *       ...
- *     }
+ * 
 {@code
+ * Query query = Query.gqlQueryBuilder("select __key__ from kind").build();
+ * QueryResults results = datastore.run(query);
+ * if (Key.class.isAssignableFrom(results.resultClass())) {
+ *   QueryResults keys = (QueryResults) results;
+ *   while (keys.hasNext()) {
+ *     Key key = keys.next();
+ *     ...
  *   }
+ * }
  * } 
* * @param the type of the result values this query will produce diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/IncompleteKey.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/IncompleteKey.java index 2ccd59e725a8..2192384ef70b 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/IncompleteKey.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/IncompleteKey.java @@ -84,6 +84,29 @@ static IncompleteKey fromPb(DatastoreV1.Key keyPb) { return new IncompleteKey(projectId, namespace, path); } + /** + * Returns the key's parent. + */ + @Override + public Key parent() { + List ancestors = ancestors(); + if (ancestors.isEmpty()) { + return null; + } + PathElement parent = ancestors.get(ancestors.size() - 1); + Key.Builder keyBuilder; + if (parent.hasName()) { + keyBuilder = Key.builder(projectId(), parent.kind(), parent.name()); + } else { + keyBuilder = Key.builder(projectId(), parent.kind(), parent.id()); + } + String namespace = namespace(); + if (namespace != null) { + keyBuilder.namespace(namespace); + } + return keyBuilder.ancestors(ancestors.subList(0, ancestors.size() - 1)).build(); + } + public static Builder builder(String projectId, String kind) { return new Builder(projectId, kind); } diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/ListValue.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/ListValue.java index 41c7e82788b5..06282a2c79d1 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/ListValue.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/ListValue.java @@ -70,17 +70,16 @@ private Builder() { super(ValueType.LIST); } - public Builder addValue(Value value) { + private void addValueHelper(Value value) { // see datastore_v1.proto definition for list_value Preconditions.checkArgument(value.type() != ValueType.LIST, "Cannot contain another list"); listBuilder.add(value); - return this; } public Builder addValue(Value first, Value... other) { - addValue(first); + addValueHelper(first); for (Value value : other) { - addValue(value); + addValueHelper(value); } return this; } diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/StructuredQuery.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/StructuredQuery.java index 15cca241e250..5892268f859c 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/StructuredQuery.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/StructuredQuery.java @@ -46,35 +46,35 @@ *

A usage example:

* *

A simple query that returns all entities for a specific kind - *

{@code
- *   Query query = Query.entityQueryBuilder().kind(kind).build();
- *   QueryResults results = datastore.run(query);
- *   while (results.hasNext()) {
- *     Entity entity = results.next();
- *     ...
- *   }
+ * 
 {@code
+ * Query query = Query.entityQueryBuilder().kind(kind).build();
+ * QueryResults results = datastore.run(query);
+ * while (results.hasNext()) {
+ *   Entity entity = results.next();
+ *   ...
+ * }
  * } 
* *

A simple key-only query of all entities for a specific kind - *

{@code
- *   Query keyOnlyQuery =  Query.keyQueryBuilder().kind(KIND1).build();
- *   QueryResults results = datastore.run(keyOnlyQuery);
- *   ...
+ * 
 {@code
+ * Query keyOnlyQuery =  Query.keyQueryBuilder().kind(KIND1).build();
+ * QueryResults results = datastore.run(keyOnlyQuery);
+ * ...
  * } 
* *

A less trivial example of a projection query that returns the first 10 results * of "age" and "name" properties (sorted and grouped by "age") with an age greater than 18 - *

{@code
- *   Query query = Query.projectionEntityQueryBuilder()
- *       .kind(kind)
- *       .projection(Projection.property("age"), Projection.first("name"))
- *       .filter(PropertyFilter.gt("age", 18))
- *       .groupBy("age")
- *       .orderBy(OrderBy.asc("age"))
- *       .limit(10)
- *       .build();
- *   QueryResults results = datastore.run(query);
- *   ...
+ * 
 {@code
+ * Query query = Query.projectionEntityQueryBuilder()
+ *     .kind(kind)
+ *     .projection(Projection.property("age"), Projection.first("name"))
+ *     .filter(PropertyFilter.gt("age", 18))
+ *     .groupBy("age")
+ *     .orderBy(OrderBy.asc("age"))
+ *     .limit(10)
+ *     .build();
+ * QueryResults results = datastore.run(query);
+ * ...
  * } 
* * @param the type of the result values this query will produce diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/Transaction.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/Transaction.java index 8089c0130f5d..78ee217f31e7 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/Transaction.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/Transaction.java @@ -30,21 +30,21 @@ * the Datastore upon {@code commit}. * A usage example: *
 {@code
- *   Transaction transaction = datastore.newTransaction();
- *   try {
- *     Entity entity = transaction.get(key);
- *     if (!entity.contains("last_name") || entity.isNull("last_name")) {
- *       String[] name = entity.getString("name").split(" ");
- *       entity = Entity.builder(entity).remove("name").set("first_name", name[0])
- *           .set("last_name", name[1]).build();
- *       transaction.update(entity);
- *       transaction.commit();
- *     }
- *   } finally {
- *     if (transaction.active()) {
- *       transaction.rollback();
- *     }
+ * Transaction transaction = datastore.newTransaction();
+ * try {
+ *   Entity entity = transaction.get(key);
+ *   if (!entity.contains("last_name") || entity.isNull("last_name")) {
+ *     String[] name = entity.getString("name").split(" ");
+ *     entity = Entity.builder(entity).remove("name").set("first_name", name[0])
+ *         .set("last_name", name[1]).build();
+ *     transaction.update(entity);
+ *     transaction.commit();
  *   }
+ * } finally {
+ *   if (transaction.active()) {
+ *     transaction.rollback();
+ *   }
+ * }
  * } 
* * @see diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DatastoreRpc.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DatastoreRpc.java similarity index 66% rename from gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DatastoreRpc.java rename to gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DatastoreRpc.java index fd916e0a1c87..002078550d1f 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DatastoreRpc.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DatastoreRpc.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.datastore.spi; import com.google.api.services.datastore.DatastoreV1.AllocateIdsRequest; import com.google.api.services.datastore.DatastoreV1.AllocateIdsResponse; @@ -35,16 +35,46 @@ */ public interface DatastoreRpc { - AllocateIdsResponse allocateIds(AllocateIdsRequest request) throws DatastoreException; + /** + * Sends an allocate IDs request. + * + * @throws DatastoreException upon failure + */ + AllocateIdsResponse allocateIds(AllocateIdsRequest request); + /** + * Sends a begin transaction request. + * + * @throws DatastoreException upon failure + */ BeginTransactionResponse beginTransaction(BeginTransactionRequest request) throws DatastoreException; - CommitResponse commit(CommitRequest request) throws DatastoreException; + /** + * Sends a commit request. + * + * @throws DatastoreException upon failure + */ + CommitResponse commit(CommitRequest request); - LookupResponse lookup(LookupRequest request) throws DatastoreException; + /** + * Sends a lookup request. + * + * @throws DatastoreException upon failure + */ + LookupResponse lookup(LookupRequest request); - RollbackResponse rollback(RollbackRequest request) throws DatastoreException; + /** + * Sends a rollback request. + * + * @throws DatastoreException upon failure + */ + RollbackResponse rollback(RollbackRequest request); - RunQueryResponse runQuery(RunQueryRequest request) throws DatastoreException; + /** + * Sends a request to run a query. + * + * @throws DatastoreException upon failure + */ + RunQueryResponse runQuery(RunQueryRequest request); } diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DatastoreRpcFactory.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DatastoreRpcFactory.java similarity index 90% rename from gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DatastoreRpcFactory.java rename to gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DatastoreRpcFactory.java index 1815dda30f5d..0979b2203037 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DatastoreRpcFactory.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DatastoreRpcFactory.java @@ -14,9 +14,10 @@ * limitations under the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.datastore.spi; import com.google.gcloud.datastore.DatastoreOptions; +import com.google.gcloud.spi.ServiceRpcFactory; /** * An interface for Datastore RPC factory. diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DefaultDatastoreRpc.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DefaultDatastoreRpc.java similarity index 92% rename from gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DefaultDatastoreRpc.java rename to gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DefaultDatastoreRpc.java index c82ff9689f68..093322fa4117 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/spi/DefaultDatastoreRpc.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/spi/DefaultDatastoreRpc.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.datastore.spi; import com.google.api.services.datastore.DatastoreV1.AllocateIdsRequest; import com.google.api.services.datastore.DatastoreV1.AllocateIdsResponse; @@ -111,8 +111,7 @@ private static DatastoreException translate( } @Override - public AllocateIdsResponse allocateIds(AllocateIdsRequest request) - throws DatastoreException { + public AllocateIdsResponse allocateIds(AllocateIdsRequest request) { try { return client.allocateIds(request); } catch (com.google.api.services.datastore.client.DatastoreException ex) { @@ -121,8 +120,7 @@ public AllocateIdsResponse allocateIds(AllocateIdsRequest request) } @Override - public BeginTransactionResponse beginTransaction(BeginTransactionRequest request) - throws DatastoreException { + public BeginTransactionResponse beginTransaction(BeginTransactionRequest request) { try { return client.beginTransaction(request); } catch (com.google.api.services.datastore.client.DatastoreException ex) { @@ -131,7 +129,7 @@ public BeginTransactionResponse beginTransaction(BeginTransactionRequest request } @Override - public CommitResponse commit(CommitRequest request) throws DatastoreException { + public CommitResponse commit(CommitRequest request) { try { return client.commit(request); } catch (com.google.api.services.datastore.client.DatastoreException ex) { @@ -140,7 +138,7 @@ public CommitResponse commit(CommitRequest request) throws DatastoreException { } @Override - public LookupResponse lookup(LookupRequest request) throws DatastoreException { + public LookupResponse lookup(LookupRequest request) { try { return client.lookup(request); } catch (com.google.api.services.datastore.client.DatastoreException ex) { @@ -149,7 +147,7 @@ public LookupResponse lookup(LookupRequest request) throws DatastoreException { } @Override - public RollbackResponse rollback(RollbackRequest request) throws DatastoreException { + public RollbackResponse rollback(RollbackRequest request) { try { return client.rollback(request); } catch (com.google.api.services.datastore.client.DatastoreException ex) { @@ -158,7 +156,7 @@ public RollbackResponse rollback(RollbackRequest request) throws DatastoreExcept } @Override - public RunQueryResponse runQuery(RunQueryRequest request) throws DatastoreException { + public RunQueryResponse runQuery(RunQueryRequest request) { try { return client.runQuery(request); } catch (com.google.api.services.datastore.client.DatastoreException ex) { diff --git a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/testing/LocalGcdHelper.java b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/testing/LocalGcdHelper.java index 7c60da50b0b0..fdb6774f810f 100644 --- a/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/testing/LocalGcdHelper.java +++ b/gcloud-java-datastore/src/main/java/com/google/gcloud/datastore/testing/LocalGcdHelper.java @@ -17,6 +17,7 @@ package com.google.gcloud.datastore.testing; import static com.google.common.base.MoreObjects.firstNonNull; +import static com.google.common.base.Preconditions.checkArgument; import static java.nio.charset.StandardCharsets.UTF_8; import com.google.common.base.Strings; @@ -85,6 +86,7 @@ public class LocalGcdHelper { private static final String GCLOUD = "gcloud"; private static final Path INSTALLED_GCD_PATH; private static final String GCD_VERSION_PREFIX = "gcd-emulator "; + private static final double DEFAULT_CONSISTENCY = 0.9; static { INSTALLED_GCD_PATH = installedGcdPath(); @@ -398,9 +400,15 @@ public LocalGcdHelper(String projectId, int port) { * All content is written to a temporary directory that will be deleted when * {@link #stop()} is called or when the program terminates) to make sure that no left-over * data from prior runs is used. + * + * @param consistency the fraction of job application attempts that will succeed, with 0.0 + * resulting in no attempts succeeding, and 1.0 resulting in all attempts succeeding. Defaults + * to 0.9. Note that setting this to 1.0 may mask incorrect assumptions about the consistency + * of non-ancestor queries; non-ancestor queries are eventually consistent. */ - public void start() throws IOException, InterruptedException { + public void start(double consistency) throws IOException, InterruptedException { // send a quick request in case we have a hanging process from a previous run + checkArgument(consistency >= 0.0 && consistency <= 1.0, "Consistency must be between 0 and 1"); sendQuitRequest(port); // Each run is associated with its own folder that is deleted once test completes. gcdPath = Files.createTempDirectory("gcd"); @@ -415,7 +423,7 @@ public void start() throws IOException, InterruptedException { } else { gcdExecutablePath = INSTALLED_GCD_PATH; } - startGcd(gcdExecutablePath); + startGcd(gcdExecutablePath, consistency); } private void downloadGcd() throws IOException { @@ -453,7 +461,8 @@ private void downloadGcd() throws IOException { } } - private void startGcd(Path executablePath) throws IOException, InterruptedException { + private void startGcd(Path executablePath, double consistency) + throws IOException, InterruptedException { // cleanup any possible data for the same project File datasetFolder = new File(gcdPath.toFile(), projectId); deleteRecurse(datasetFolder.toPath()); @@ -486,7 +495,8 @@ private void startGcd(Path executablePath) throws IOException, InterruptedExcept startProcess = CommandWrapper.create() .command(gcdAbsolutePath.toString(), "start", "--testing", "--allow_remote_shutdown", - "--port=" + Integer.toString(port), projectId) + "--port=" + Integer.toString(port), "--consistency=" + Double.toString(consistency), + projectId) .directory(gcdPath) .start(); processReader = ProcessStreamReader.start(startProcess.getInputStream()); @@ -526,6 +536,7 @@ private static void extractFile(ZipInputStream zipIn, File filePath) throws IOEx public static boolean sendQuitRequest(int port) { StringBuilder result = new StringBuilder(); + String shutdownMsg = "Shutting down local server"; try { URL url = new URL("http", "localhost", port, "/_ah/admin/quit"); HttpURLConnection con = (HttpURLConnection) url.openConnection(); @@ -537,13 +548,13 @@ public static boolean sendQuitRequest(int port) { out.flush(); InputStream in = con.getInputStream(); int currByte = 0; - while ((currByte = in.read()) != -1) { + while ((currByte = in.read()) != -1 && result.length() < shutdownMsg.length()) { result.append(((char) currByte)); } } catch (IOException ignore) { // ignore } - return result.toString().startsWith("Shutting down local server"); + return result.toString().startsWith(shutdownMsg); } public void stop() throws IOException, InterruptedException { @@ -578,10 +589,10 @@ public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IO }); } - public static LocalGcdHelper start(String projectId, int port) + public static LocalGcdHelper start(String projectId, int port, double consistency) throws IOException, InterruptedException { LocalGcdHelper helper = new LocalGcdHelper(projectId, port); - helper.start(); + helper.start(consistency); return helper; } @@ -593,7 +604,9 @@ public static void main(String... args) throws IOException, InterruptedException switch (action) { case "START": if (!isActive(DEFAULT_PROJECT_ID, port)) { - LocalGcdHelper helper = start(DEFAULT_PROJECT_ID, port); + double consistency = parsedArgs.get("consistency") == null + ? DEFAULT_CONSISTENCY : Double.parseDouble(parsedArgs.get("consistency")); + LocalGcdHelper helper = start(DEFAULT_PROJECT_ID, port, consistency); try (FileWriter writer = new FileWriter(".local_gcd_helper")) { writer.write(helper.gcdPath.toAbsolutePath().toString() + System.lineSeparator()); writer.write(Integer.toString(port)); diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/BaseEntityTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/BaseEntityTest.java index 5ece01508d3a..a69ea5e20e3b 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/BaseEntityTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/BaseEntityTest.java @@ -67,6 +67,16 @@ public void setUp() { builder.set("list1", NullValue.of(), StringValue.of("foo")); builder.set("list2", ImmutableList.of(LongValue.of(10), DoubleValue.of(2))); builder.set("list3", Collections.singletonList(BooleanValue.of(true))); + builder.set( + "blobList", BLOB, Blob.copyFrom(new byte[] {3, 4}), Blob.copyFrom(new byte[] {5, 6})); + builder.set("booleanList", true, false, true); + builder.set("dateTimeList", DateTime.now(), DateTime.now(), DateTime.now()); + builder.set("doubleList", 12.3, 4.56, .789); + builder.set("keyList", KEY, Key.builder("ds2", "k2", "n2").build(), + Key.builder("ds3", "k3", "n3").build()); + builder.set("entityList", ENTITY, PARTIAL_ENTITY); + builder.set("stringList", "s1", "s2", "s3"); + builder.set("longList", 1, 23, 456); } @Test @@ -183,6 +193,17 @@ public void testGetList() throws Exception { assertEquals(Boolean.TRUE, list.get(0).get()); entity = builder.set("list1", ListValue.of(list)).build(); assertEquals(list, entity.getList("list1")); + List> stringList = entity.getList("stringList"); + assertEquals( + ImmutableList.of(StringValue.of("s1"), StringValue.of("s2"), StringValue.of("s3")), + stringList); + List> doubleList = entity.getList("doubleList"); + assertEquals( + ImmutableList.of(DoubleValue.of(12.3), DoubleValue.of(4.56), DoubleValue.of(.789)), + doubleList); + List entityList = entity.getList("entityList"); + assertEquals( + ImmutableList.of(EntityValue.of(ENTITY), EntityValue.of(PARTIAL_ENTITY)), entityList); } @Test @@ -198,7 +219,9 @@ public void testGetBlob() throws Exception { public void testNames() throws Exception { Set names = ImmutableSet.builder() .add("string", "stringValue", "boolean", "double", "long", "list1", "list2", "list3") - .add("entity", "partialEntity", "null", "dateTime", "blob", "key") + .add("entity", "partialEntity", "null", "dateTime", "blob", "key", "blobList") + .add("booleanList", "dateTimeList", "doubleList", "keyList", "entityList", "stringList") + .add("longList") .build(); BaseEntity entity = builder.build(); assertEquals(names, entity.names()); diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/BaseKeyTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/BaseKeyTest.java index 8615ee025bd1..43db4695b191 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/BaseKeyTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/BaseKeyTest.java @@ -50,6 +50,11 @@ protected BaseKey build() { protected Object fromPb(byte[] bytesPb) throws InvalidProtocolBufferException { return null; } + + @Override + protected BaseKey parent() { + return null; + } }; } } diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreExceptionTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreExceptionTest.java index 4d62224172f9..f7bdcb89bcec 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreExceptionTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreExceptionTest.java @@ -78,7 +78,8 @@ public void testDatastoreException() throws Exception { @Test public void testTranslateAndThrow() throws Exception { DatastoreException cause = new DatastoreException(503, "message", "UNAVAILABLE"); - RetryHelper.RetryHelperException exceptionMock = createMock(RetryHelper.RetryHelperException.class); + RetryHelper.RetryHelperException exceptionMock = + createMock(RetryHelper.RetryHelperException.class); expect(exceptionMock.getCause()).andReturn(cause).times(2); replay(exceptionMock); try { diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreOptionsTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreOptionsTest.java index 284a9d322793..1d188c7f4e94 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreOptionsTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreOptionsTest.java @@ -22,9 +22,9 @@ import static org.junit.Assert.assertSame; import static org.junit.Assert.assertTrue; +import com.google.gcloud.datastore.spi.DatastoreRpc; +import com.google.gcloud.datastore.spi.DatastoreRpcFactory; import com.google.gcloud.datastore.testing.LocalGcdHelper; -import com.google.gcloud.spi.DatastoreRpc; -import com.google.gcloud.spi.DatastoreRpcFactory; import org.easymock.EasyMock; import org.junit.Before; diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreTest.java index e9eed027e8e0..e3829a2e71ce 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/DatastoreTest.java @@ -31,14 +31,16 @@ import com.google.api.services.datastore.DatastoreV1.RunQueryRequest; import com.google.api.services.datastore.DatastoreV1.RunQueryResponse; import com.google.common.collect.Iterators; +import com.google.common.collect.Lists; +import com.google.gcloud.AuthCredentials; import com.google.gcloud.RetryParams; import com.google.gcloud.datastore.Query.ResultType; import com.google.gcloud.datastore.StructuredQuery.OrderBy; import com.google.gcloud.datastore.StructuredQuery.Projection; import com.google.gcloud.datastore.StructuredQuery.PropertyFilter; +import com.google.gcloud.datastore.spi.DatastoreRpc; +import com.google.gcloud.datastore.spi.DatastoreRpcFactory; import com.google.gcloud.datastore.testing.LocalGcdHelper; -import com.google.gcloud.spi.DatastoreRpc; -import com.google.gcloud.spi.DatastoreRpcFactory; import com.google.protobuf.ByteString; import org.easymock.EasyMock; @@ -89,8 +91,8 @@ public class DatastoreTest { FullEntity.builder(INCOMPLETE_KEY2).set("str", STR_VALUE).set("bool", BOOL_VALUE) .set("list", LIST_VALUE1).build(); private static final FullEntity PARTIAL_ENTITY2 = - FullEntity.builder(PARTIAL_ENTITY1).remove("str").set("bool", true). - set("list", LIST_VALUE1.get()).build(); + FullEntity.builder(PARTIAL_ENTITY1).remove("str").set("bool", true) + .set("list", LIST_VALUE1.get()).build(); private static final FullEntity PARTIAL_ENTITY3 = FullEntity.builder(PARTIAL_ENTITY1).key(IncompleteKey.builder(PROJECT_ID, KIND3).build()) .build(); @@ -118,7 +120,7 @@ public class DatastoreTest { @BeforeClass public static void beforeClass() throws IOException, InterruptedException { if (!LocalGcdHelper.isActive(PROJECT_ID, PORT)) { - gcdHelper = LocalGcdHelper.start(PROJECT_ID, PORT); + gcdHelper = LocalGcdHelper.start(PROJECT_ID, PORT, 1.0); } } @@ -127,6 +129,7 @@ public void setUp() { options = DatastoreOptions.builder() .projectId(PROJECT_ID) .host("http://localhost:" + PORT) + .authCredentials(AuthCredentials.noAuth()) .retryParams(RetryParams.noRetries()) .build(); datastore = options.service(); @@ -471,9 +474,13 @@ public void testQueryPaginationWithLimit() throws DatastoreException { EasyMock.expect(rpcFactoryMock.create(EasyMock.anyObject(DatastoreOptions.class))) .andReturn(rpcMock); List responses = buildResponsesForQueryPaginationWithLimit(); - for (int i = 0; i < responses.size(); i++) { + List endCursors = Lists.newArrayListWithCapacity(responses.size()); + for (RunQueryResponse response : responses) { EasyMock.expect(rpcMock.runQuery(EasyMock.anyObject(RunQueryRequest.class))) - .andReturn(responses.get(i)); + .andReturn(response); + if (response.getBatch().getMoreResults() != QueryResultBatch.MoreResultsType.NOT_FINISHED) { + endCursors.add(response.getBatch().getEndCursor()); + } } EasyMock.replay(rpcFactoryMock, rpcMock); Datastore mockDatastore = options.toBuilder() @@ -483,6 +490,7 @@ public void testQueryPaginationWithLimit() throws DatastoreException { .service(); int limit = 2; int totalCount = 0; + Iterator cursorIter = endCursors.iterator(); StructuredQuery query = Query.entityQueryBuilder().limit(limit).build(); while (true) { QueryResults results = mockDatastore.run(query); @@ -492,6 +500,9 @@ public void testQueryPaginationWithLimit() throws DatastoreException { resultCount++; totalCount++; } + assertTrue(cursorIter.hasNext()); + Cursor expectedEndCursor = Cursor.copyFrom(cursorIter.next().toByteArray()); + assertEquals(expectedEndCursor, results.cursorAfter()); if (resultCount < limit) { break; } @@ -505,19 +516,20 @@ private List buildResponsesForQueryPaginationWithLimit() { Entity entity4 = Entity.builder(KEY4).set("value", StringValue.of("value")).build(); Entity entity5 = Entity.builder(KEY5).set("value", "value").build(); datastore.add(ENTITY3, entity4, entity5); + DatastoreRpc datastoreRpc = datastore.options().rpc(); List responses = new ArrayList<>(); Query query = Query.entityQueryBuilder().build(); RunQueryRequest.Builder requestPb = RunQueryRequest.newBuilder(); query.populatePb(requestPb); QueryResultBatch queryResultBatchPb = RunQueryResponse.newBuilder() - .mergeFrom(((DatastoreImpl) datastore).runQuery(requestPb.build())) + .mergeFrom(datastoreRpc.runQuery(requestPb.build())) .getBatch(); QueryResultBatch queryResultBatchPb1 = QueryResultBatch.newBuilder() .mergeFrom(queryResultBatchPb) .setMoreResults(QueryResultBatch.MoreResultsType.NOT_FINISHED) .clearEntityResult() .addAllEntityResult(queryResultBatchPb.getEntityResultList().subList(0, 1)) - .setEndCursor(queryResultBatchPb.getEntityResultList().get(0).getCursor()) + .setEndCursor(ByteString.copyFromUtf8("a")) .build(); responses.add(RunQueryResponse.newBuilder().setBatch(queryResultBatchPb1).build()); QueryResultBatch queryResultBatchPb2 = QueryResultBatch.newBuilder() @@ -534,7 +546,7 @@ private List buildResponsesForQueryPaginationWithLimit() { .setMoreResults(QueryResultBatch.MoreResultsType.MORE_RESULTS_AFTER_LIMIT) .clearEntityResult() .addAllEntityResult(queryResultBatchPb.getEntityResultList().subList(2, 4)) - .setEndCursor(queryResultBatchPb.getEntityResultList().get(3).getCursor()) + .setEndCursor(ByteString.copyFromUtf8("b")) .build(); responses.add(RunQueryResponse.newBuilder().setBatch(queryResultBatchPb3).build()); QueryResultBatch queryResultBatchPb4 = QueryResultBatch.newBuilder() @@ -542,7 +554,7 @@ private List buildResponsesForQueryPaginationWithLimit() { .setMoreResults(QueryResultBatch.MoreResultsType.NO_MORE_RESULTS) .clearEntityResult() .addAllEntityResult(queryResultBatchPb.getEntityResultList().subList(4, 5)) - .setEndCursor(queryResultBatchPb.getEntityResultList().get(4).getCursor()) + .setEndCursor(ByteString.copyFromUtf8("c")) .build(); responses.add(RunQueryResponse.newBuilder().setBatch(queryResultBatchPb4).build()); return responses; diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/IncompleteKeyTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/IncompleteKeyTest.java index 7edbf133d330..acd1dfd3c9e3 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/IncompleteKeyTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/IncompleteKeyTest.java @@ -17,29 +17,47 @@ package com.google.gcloud.datastore; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; import static org.junit.Assert.assertTrue; +import org.junit.Before; import org.junit.Test; public class IncompleteKeyTest { + private static IncompleteKey pk1, pk2; + private static Key parent1; + + @Before + public void setUp() { + pk1 = IncompleteKey.builder("ds", "kind1").build(); + parent1 = Key.builder("ds", "kind2", 10).namespace("ns").build(); + pk2 = IncompleteKey.builder(parent1, "kind3").build(); + } + @Test public void testBuilders() throws Exception { - IncompleteKey pk1 = IncompleteKey.builder("ds", "kind1").build(); assertEquals("ds", pk1.projectId()); assertEquals("kind1", pk1.kind()); assertTrue(pk1.ancestors().isEmpty()); - Key parent = Key.builder("ds", "kind2", 10).build(); - IncompleteKey pk2 = IncompleteKey.builder(parent, "kind3").build(); assertEquals("ds", pk2.projectId()); assertEquals("kind3", pk2.kind()); - assertEquals(parent.path(), pk2.ancestors()); + assertEquals(parent1.path(), pk2.ancestors()); assertEquals(pk2, IncompleteKey.builder(pk2).build()); IncompleteKey pk3 = IncompleteKey.builder(pk2).kind("kind4").build(); assertEquals("ds", pk3.projectId()); assertEquals("kind4", pk3.kind()); - assertEquals(parent.path(), pk3.ancestors()); + assertEquals(parent1.path(), pk3.ancestors()); + } + + @Test + public void testParent() { + assertNull(pk1.parent()); + assertEquals(parent1, pk2.parent()); + Key parent2 = Key.builder("ds", "kind3", "name").namespace("ns").build(); + IncompleteKey pk3 = IncompleteKey.builder(parent2, "kind3").build(); + assertEquals(parent2, pk3.parent()); } } diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/LocalGcdHelperTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/LocalGcdHelperTest.java index 40ea62c5a7e0..5d761a713506 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/LocalGcdHelperTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/LocalGcdHelperTest.java @@ -49,7 +49,7 @@ public void testFindAvailablePort() { @Test public void testSendQuitRequest() throws IOException, InterruptedException { - LocalGcdHelper gcdHelper = LocalGcdHelper.start(PROJECT_ID, PORT); + LocalGcdHelper gcdHelper = LocalGcdHelper.start(PROJECT_ID, PORT, 0.75); assertTrue(LocalGcdHelper.sendQuitRequest(PORT)); long timeoutMillis = 30000; long startTime = System.currentTimeMillis(); @@ -64,7 +64,7 @@ public void testSendQuitRequest() throws IOException, InterruptedException { @Test public void testStartStop() throws IOException, InterruptedException { - LocalGcdHelper gcdHelper = LocalGcdHelper.start(PROJECT_ID, PORT); + LocalGcdHelper gcdHelper = LocalGcdHelper.start(PROJECT_ID, PORT, 0.75); assertFalse(LocalGcdHelper.isActive("wrong-project-id", PORT)); assertTrue(LocalGcdHelper.isActive(PROJECT_ID, PORT)); gcdHelper.stop(); diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/SerializationTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/SerializationTest.java index 3976be2cc383..b9e78800ffab 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/SerializationTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/SerializationTest.java @@ -17,28 +17,17 @@ package com.google.gcloud.datastore; import static java.nio.charset.StandardCharsets.UTF_8; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotSame; import com.google.api.services.datastore.DatastoreV1; -import com.google.common.collect.ImmutableMultimap; -import com.google.common.collect.Multimap; import com.google.gcloud.AuthCredentials; -import com.google.gcloud.RetryParams; +import com.google.gcloud.BaseSerializationTest; +import com.google.gcloud.Restorable; import com.google.gcloud.datastore.StructuredQuery.CompositeFilter; import com.google.gcloud.datastore.StructuredQuery.OrderBy; import com.google.gcloud.datastore.StructuredQuery.Projection; import com.google.gcloud.datastore.StructuredQuery.PropertyFilter; -import org.junit.Test; - -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.IOException; -import java.io.ObjectInputStream; -import java.io.ObjectOutputStream; - -public class SerializationTest { +public class SerializationTest extends BaseSerializationTest { private static final IncompleteKey INCOMPLETE_KEY1 = IncompleteKey.builder("ds", "k").ancestors(PathElement.of("p", 1)).build(); @@ -113,83 +102,31 @@ public class SerializationTest { .addValue(new NullValue()) .build(); private static final ProjectionEntity PROJECTION_ENTITY = ProjectionEntity.fromPb(ENTITY1.toPb()); + private static final DatastoreException DATASTORE_EXCEPTION = + new DatastoreException(42, "message", "reason"); - @SuppressWarnings("rawtypes") - private static final Multimap TYPE_TO_VALUES = - ImmutableMultimap.builder() - .put(ValueType.NULL, NULL_VALUE) - .put(ValueType.KEY, KEY_VALUE) - .put(ValueType.STRING, STRING_VALUE) - .putAll(ValueType.ENTITY, EMBEDDED_ENTITY_VALUE1, EMBEDDED_ENTITY_VALUE2, - EMBEDDED_ENTITY_VALUE3) - .put(ValueType.LIST, LIST_VALUE) - .put(ValueType.LONG, LONG_VALUE) - .put(ValueType.DOUBLE, DOUBLE_VALUE) - .put(ValueType.BOOLEAN, BOOLEAN_VALUE) - .put(ValueType.DATE_TIME, DATE_AND_TIME_VALUE) - .put(ValueType.BLOB, BLOB_VALUE) - .put(ValueType.RAW_VALUE, RAW_VALUE) - .build(); - - @Test - public void testServiceOptions() throws Exception { + @Override + protected java.io.Serializable[] serializableObjects() { DatastoreOptions options = DatastoreOptions.builder() .authCredentials(AuthCredentials.createForAppEngine()) .normalizeDataset(false) .projectId("ds1") .build(); - DatastoreOptions serializedCopy = serializeAndDeserialize(options); - assertEquals(options, serializedCopy); - - options = options.toBuilder() + DatastoreOptions otherOptions = options.toBuilder() .namespace("ns1") - .retryParams(RetryParams.defaultInstance()) .authCredentials(null) .force(true) .build(); - serializedCopy = serializeAndDeserialize(options); - assertEquals(options, serializedCopy); - } - - @Test - public void testValues() throws Exception { - for (ValueType valueType : ValueType.values()) { - for (Value value : TYPE_TO_VALUES.get(valueType)) { - Value copy = serializeAndDeserialize(value); - assertEquals(value, value); - assertEquals(value, copy); - assertNotSame(value, copy); - assertEquals(copy, copy); - assertEquals(value.get(), copy.get()); - } - } - } - - @Test - public void testTypes() throws Exception { - Serializable[] types = { KEY1, KEY2, INCOMPLETE_KEY1, INCOMPLETE_KEY2, ENTITY1, ENTITY2, - ENTITY3, EMBEDDED_ENTITY, PROJECTION_ENTITY, DATE_TIME1, BLOB1, CURSOR1, GQL1, GQL2, - QUERY1, QUERY2, QUERY3}; - for (Serializable obj : types) { - Object copy = serializeAndDeserialize(obj); - assertEquals(obj, obj); - assertEquals(obj, copy); - assertNotSame(obj, copy); - assertEquals(copy, copy); - } + return new java.io.Serializable[]{KEY1, KEY2, INCOMPLETE_KEY1, INCOMPLETE_KEY2, ENTITY1, + ENTITY2, ENTITY3, EMBEDDED_ENTITY, PROJECTION_ENTITY, DATE_TIME1, BLOB1, CURSOR1, GQL1, + GQL2, QUERY1, QUERY2, QUERY3, NULL_VALUE, KEY_VALUE, STRING_VALUE, EMBEDDED_ENTITY_VALUE1, + EMBEDDED_ENTITY_VALUE2, EMBEDDED_ENTITY_VALUE3, LIST_VALUE, LONG_VALUE, DOUBLE_VALUE, + BOOLEAN_VALUE, DATE_AND_TIME_VALUE, BLOB_VALUE, RAW_VALUE, DATASTORE_EXCEPTION, options, + otherOptions}; } - private T serializeAndDeserialize(T obj) - throws IOException, ClassNotFoundException { - ByteArrayOutputStream bytes = new ByteArrayOutputStream(); - try (ObjectOutputStream output = new ObjectOutputStream(bytes)) { - output.writeObject(obj); - } - try (ObjectInputStream input = - new ObjectInputStream(new ByteArrayInputStream(bytes.toByteArray()))) { - @SuppressWarnings("unchecked") - T result = (T) input.readObject(); - return result; - } + @Override + protected Restorable[] restorableObjects() { + return null; } } diff --git a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/StructuredQueryTest.java b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/StructuredQueryTest.java index b0d188cae16e..4b6589efd723 100644 --- a/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/StructuredQueryTest.java +++ b/gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/StructuredQueryTest.java @@ -40,7 +40,8 @@ public class StructuredQueryTest { private static final Cursor END_CURSOR = Cursor.copyFrom(new byte[] {10}); private static final int OFFSET = 42; private static final Integer LIMIT = 43; - private static final Filter FILTER = CompositeFilter.and(PropertyFilter.gt("p1", 10), PropertyFilter.eq("a", "v")); + private static final Filter FILTER = + CompositeFilter.and(PropertyFilter.gt("p1", 10), PropertyFilter.eq("a", "v")); private static final OrderBy ORDER_BY_1 = OrderBy.asc("p2"); private static final OrderBy ORDER_BY_2 = OrderBy.desc("p3"); private static final List ORDER_BY = ImmutableList.of(ORDER_BY_1, ORDER_BY_2); diff --git a/gcloud-java-dns/README.md b/gcloud-java-dns/README.md new file mode 100644 index 000000000000..d2e4c85b3b76 --- /dev/null +++ b/gcloud-java-dns/README.md @@ -0,0 +1,393 @@ +Google Cloud Java Client for DNS +================================ + +Java idiomatic client for [Google Cloud DNS] (https://cloud.google.com/dns/). + +[![Build Status](https://travis-ci.org/GoogleCloudPlatform/gcloud-java.svg?branch=master)](https://travis-ci.org/GoogleCloudPlatform/gcloud-java) +[![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) +[![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-dns.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-dns.svg) +[![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) + +- [Homepage] (https://googlecloudplatform.github.io/gcloud-java/) +- [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/dns/package-summary.html) + +> Note: This client is a work-in-progress, and may occasionally +> make backwards-incompatible changes. + +Quickstart +---------- +If you are using Maven, add this to your pom.xml file +```xml + + com.google.gcloud + gcloud-java-dns + 0.1.5 + +``` +If you are using Gradle, add this to your dependencies +```Groovy +compile 'com.google.gcloud:gcloud-java-dns:0.1.5' +``` +If you are using SBT, add this to your dependencies +```Scala +libraryDependencies += "com.google.gcloud" % "gcloud-java-dns" % "0.1.5" +``` + +Example Application +------------------- + +[`DnsExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/DnsExample.java) +is a simple command line interface that provides some of Google Cloud DNS's functionality. Read +more about using the application on the +[`DnsExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/dns/DnsExample.html). + +Authentication +-------------- + +See the [Authentication](https://github.com/GoogleCloudPlatform/gcloud-java#authentication) section +in the base directory's README. + +About Google Cloud DNS +-------------------------- + +[Google Cloud DNS][cloud-dns] is a scalable, reliable and managed authoritative Domain Name System +(DNS) service running on the same infrastructure as Google. It has low latency, high availability +and is a cost-effective way to make your applications and services available to your users. + +See the [Google Cloud DNS docs][dns-activate] for more details on how to activate +Cloud DNS for your project. + +See the [``gcloud-java-dns`` API documentation][dns-api] to learn how to interact +with the Cloud DNS using this client Library. + +Getting Started +--------------- +#### Prerequisites +For this tutorial, you will need a [Google Developers Console](https://console.developers.google.com/) +project with the DNS API enabled. You will need to [enable billing](https://support.google.com/cloud/answer/6158867?hl=en) +to use Google Cloud DNS. [Follow these instructions](https://cloud.google.com/docs/authentication#preparation) +to get your project set up. You will also need to set up the local development environment by +[installing the Google Cloud SDK](https://cloud.google.com/sdk/) and running the following commands +in command line: `gcloud auth login` and `gcloud config set project [YOUR PROJECT ID]`. + +#### Installation and setup +You'll need to obtain the `gcloud-java-dns` library. See the [Quickstart](#quickstart) section to +add `gcloud-java-dns` as a dependency in your code. + +#### Creating an authorized service object +To make authenticated requests to Google Cloud DNS, you must create a service object with +credentials. You can then make API calls by calling methods on the DNS service object. The simplest +way to authenticate is to use [Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials). +These credentials are automatically inferred from your environment, so you only need the following +code to create your service object: + +```java +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsOptions; + +Dns dns = DnsOptions.defaultInstance().service(); +``` + +For other authentication options, see the [Authentication](https://github.com/GoogleCloudPlatform/gcloud-java#authentication) page. + +#### Managing Zones +Record sets in `gcloud-java-dns` are managed inside containers called "zones". `ZoneInfo` is a class +which encapsulates metadata that describe a zone in Google Cloud DNS. `Zone`, a subclass of `ZoneInfo`, adds service-related +functionality over `ZoneInfo`. + +*Important: Zone names must be unique to the project. If you choose a zone name that already +exists within your project, you'll get a helpful error message telling you to choose another name. In the code below, +replace "my-unique-zone" with a unique zone name. See more about naming rules [here](https://cloud.google.com/dns/api/v1/managedZones#name).* + +In this code snippet, we create a new zone to manage record sets for domain `someexampledomain.com.` + +*Important: The service may require that you verify ownership of the domain for which you are creating a zone. +Hence, we recommend that you do so beforehand. You can verify ownership of +a domain name [here](https://www.google.com/webmasters/verification/home). Note that Cloud DNS +requires fully qualified domain names which must end with a period.* + +Add the following imports at the top of your file: + +```java +import com.google.gcloud.dns.Zone; +import com.google.gcloud.dns.ZoneInfo; +``` + +Then add the following code to create a zone. + +```java +// Create a zone metadata object +String zoneName = "my-unique-zone"; // Change this zone name which is unique within your project +String domainName = "someexampledomain.com."; // Change this to a domain which you own +String description = "This is a gcloud-java-dns sample zone."; +ZoneInfo zoneInfo = ZoneInfo.of(zoneName, domainName, description); + +// Create zone in Google Cloud DNS +Zone zone = dns.create(zoneInfo); +System.out.printf("Zone was created and assigned ID %s.%n", zone.id()); +``` + +You now have an empty zone hosted in Google Cloud DNS which is ready to be populated with +record sets for domain name `someexampledomain.com.` Upon creating the zone, the cloud service +assigned a set of DNS servers to host records for this zone and +created the required SOA and NS records for the domain. The following snippet prints the list of servers +assigned to the zone created above. First, import + +```java +import java.util.List; +``` + +and then add + +```java +// Print assigned name servers +List nameServers = zone.nameServers(); +for(String nameServer : nameServers) { + System.out.println(nameServer); +} +``` + +You can now instruct your domain registrar to [update your domain name servers] (https://cloud.google.com/dns/update-name-servers). +As soon as this happens and the change propagates through cached values in DNS resolvers, +all the DNS queries will be directed to and answered by the Google Cloud DNS service. + +#### Creating Record Sets +Now that we have a zone, we can add some record sets. The record sets held within zones are +modified by "change requests". In this example, we create and apply a change request to +our zone that creates a record set of type A and points URL www.someexampledomain.com to +IP address 12.13.14.15. Start by adding + +```java +import com.google.gcloud.dns.ChangeRequestInfo; +import com.google.gcloud.dns.RecordSet; + +import java.util.concurrent.TimeUnit; +``` + +and proceed with: + +```java +// Prepare a www.someexampledomain.com. type A record set with ttl of 24 hours +String ip = "12.13.14.15"; +RecordSet toCreate = RecordSet.builder("www." + zone.dnsName(), RecordSet.Type.A) + .ttl(24, TimeUnit.HOURS) + .addRecord(ip) + .build(); + +// Make a change +ChangeRequestInfo changeRequest = ChangeRequestInfo.builder().add(toCreate).build(); + +// Build and apply the change request to our zone +changeRequest = zone.applyChangeRequest(changeRequest); +``` + +The `addRecord` method of `RecordSet.Builder` accepts records in the form of +strings. The format of the strings depends on the type of the record sets to be added. +More information on the supported record set types and record formats can be found [here](https://cloud.google.com/dns/what-is-cloud-dns#supported_record_types). + +If you already have a record set, Cloud DNS will return an error upon an attempt to create a duplicate of it. +You can modify the code above to create a record set or update it if it already exists by making the +following adjustment in your imports + +```java +import java.util.Iterator; +``` + +and in the code + +```java +// Make a change +ChangeRequestInfo.Builder changeBuilder = ChangeRequestInfo.builder().add(toCreate); + +// Verify the type A record does not exist yet. +// If it does exist, we will overwrite it with our prepared record. +Iterator recordSetIterator = zone.listRecordSets().iterateAll(); +while (recordSetIterator.hasNext()) { + RecordSet current = recordSetIterator.next(); + if (toCreate.name().equals(current.name()) && toCreate.type().equals(current.type())) { + changeBuilder.delete(current); + } +} + +// Build and apply the change request to our zone +ChangeRequestInfo changeRequest = changeBuilder.build(); +zone.applyChangeRequest(changeRequest); +``` +You can find more information about changes in the [Cloud DNS documentation] (https://cloud.google.com/dns/what-is-cloud-dns#cloud_dns_api_concepts). + +When the change request is applied, it is registered with the Cloud DNS service for processing. We +can wait for its completion as follows: + +```java +while (ChangeRequestInfo.Status.PENDING.equals(changeRequest.status())) { + try { + Thread.sleep(500L); + } catch (InterruptedException e) { + System.err.println("The thread was interrupted while waiting..."); + } + changeRequest = dns.getChangeRequest(zone.name(), changeRequest.id()); +} +System.out.println("The change request has been applied."); +``` + +Change requests are applied atomically to all the assigned DNS servers at once. Note that when this +happens, it may still take a while for the change to be registered by the DNS cache resolvers. +See more on this topic [here](https://cloud.google.com/dns/monitoring). + +#### Listing Zones and Record Sets +Suppose that you have added more zones and record sets, and now you want to list them. +First, import the following (unless you have done so in the previous section): + +```java +import java.util.Iterator; +``` + +Then add the following code to list all your zones and record sets. + +```java +// List all your zones +Iterator zoneIterator = dns.listZones().iterateAll(); +int counter = 1; +while (zoneIterator.hasNext()) { + System.out.printf("#%d.: %s%n%n", counter, zoneIterator.next()); + counter++; +} + +// List the record sets in a particular zone +recordSetIterator = zone.listRecordSets().iterateAll(); +System.out.println(String.format("Record sets inside %s:", zone.name())); +while (recordSetIterator.hasNext()) { + System.out.println(recordSetIterator.next()); +} +``` + +You can also list the history of change requests that were applied to a zone. +First add: + +```java +import java.util.ChangeRequest; +``` + +and then: + +```java + +// List the change requests applied to a particular zone +Iterator changeIterator = zone.listChangeRequests().iterateAll(); +System.out.println(String.format("The history of changes in %s:", zone.name())); +while (changeIterator.hasNext()) { + System.out.println(changeIterator.next()); +} +``` + +#### Deleting Zones + +If you no longer want to host a zone in Cloud DNS, you can delete it. +First, you need to empty the zone by deleting all its records except for the default SOA and NS record sets. + +```java +// Make a change for deleting the record sets +changeBuilder = ChangeRequestInfo.builder(); +while (recordIterator.hasNext()) { + RecordSet current = recordIterator.next(); + // SOA and NS records cannot be deleted + if (!RecordSet.Type.SOA.equals(current.type()) && !RecordSet.Type.NS.equals(current.type())) { + changeBuilder.delete(current); + } +} + +// Build and apply the change request to our zone if it contains records to delete +ChangeRequestInfo changeRequest = changeBuilder.build(); +if (!changeRequest.deletions().isEmpty()) { + changeRequest = dns.applyChangeRequest(zoneName, changeRequest); + + // Wait for change to finish, but save data traffic by transferring only ID and status + Dns.ChangeRequestOption option = + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS); + while (ChangeRequestInfo.Status.PENDING.equals(changeRequest.status())) { + System.out.println("Waiting for change to complete. Going to sleep for 500ms..."); + try { + Thread.sleep(500); + } catch (InterruptedException e) { + System.err.println("The thread was interrupted while waiting for change request to be " + + "processed."); + } + + // Update the change, but fetch only change ID and status + changeRequest = dns.getChangeRequest(zoneName, changeRequest.id(), option); + } +} + +// Delete the zone +boolean result = dns.delete(zoneName); +if (result) { + System.out.println("Zone was deleted."); +} else { + System.out.println("Zone was not deleted because it does not exist."); +} +``` + +#### Complete Source Code + +We composed some of the aforementioned snippets into complete executable code samples. In +[CreateZones.java](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateZone.java) +we create a zone. In [CreateOrUpdateRecordSets.java](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateOrUpdateRecordSets.java) +we create a type A record set for a zone, or update an existing type A record set to a new IP address. We +demonstrate how to delete a zone in [DeleteZone.java](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/DeleteZone.java). +Finally, in [ManipulateZonesAndRecordSets.java](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/ManipulateZonesAndRecordSets.java) +we assemble all the code snippets together and create zone, create or update a record set, list zones, list record sets, list changes, and +delete a zone. The applications assume that they are running on Compute Engine or from your own desktop. To run any of these examples on App +Engine, simply move the code from the main method to your application's servlet class and change the +print statements to display on your webpage. + +Troubleshooting +--------------- + +To get help, follow the `gcloud-java` links in the `gcloud-*` [shared Troubleshooting document](https://github.com/GoogleCloudPlatform/gcloud-common/blob/master/troubleshooting/readme.md#troubleshooting). + +Java Versions +------------- + +Java 7 or above is required for using this client. + +Testing +------- + +This library has tools to help make tests for code using Cloud DNS. + +See [TESTING] to read more about testing. + +Versioning +---------- + +This library follows [Semantic Versioning] (http://semver.org/). + +It is currently in major version zero (``0.y.z``), which means that anything +may change at any time and the public API should not be considered +stable. + +Contributing +------------ + +Contributions to this library are always welcome and highly encouraged. + +See `gcloud-java`'s [CONTRIBUTING] documentation and the `gcloud-*` [shared documentation](https://github.com/GoogleCloudPlatform/gcloud-common/blob/master/contributing/readme.md#how-to-contribute-to-gcloud) for more information on how to get started. + +Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See [Code of Conduct][code-of-conduct] for more information. + +License +------- + +Apache 2.0 - See [LICENSE] for more information. + + +[CONTRIBUTING]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CONTRIBUTING.md +[code-of-conduct]:https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/CODE_OF_CONDUCT.md#contributor-code-of-conduct +[LICENSE]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/LICENSE +[TESTING]: https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/TESTING.md#testing-code-that-uses-storage +[cloud-platform]: https://cloud.google.com/ + +[cloud-dns]: https://cloud.google.com/dns/ +[dns-api]: http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/dns/package-summary.html +[dns-activate]:https://cloud.google.com/dns/getting-started#prerequisites diff --git a/gcloud-java-dns/pom.xml b/gcloud-java-dns/pom.xml new file mode 100644 index 000000000000..b55200b8fc7d --- /dev/null +++ b/gcloud-java-dns/pom.xml @@ -0,0 +1,63 @@ + + + 4.0.0 + com.google.gcloud + gcloud-java-dns + jar + GCloud Java DNS + + Java idiomatic client for Google Cloud DNS. + + + com.google.gcloud + gcloud-java-pom + 0.1.6-SNAPSHOT + + + gcloud-java-dns + + + + ${project.groupId} + gcloud-java-core + ${project.version} + + + com.google.apis + google-api-services-dns + v1-rev7-1.21.0 + compile + + + com.google.guava + guava-jdk5 + + + com.google.api-client + google-api-client + + + + + ${project.groupId} + gcloud-java-core + ${project.version} + test-jar + test + + + junit + junit + 4.12 + test + + + org.easymock + easymock + 3.3 + test + + + diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/AbstractOption.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/AbstractOption.java new file mode 100644 index 000000000000..e12f7412e687 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/AbstractOption.java @@ -0,0 +1,70 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.common.base.MoreObjects; +import com.google.gcloud.dns.spi.DnsRpc; + +import java.io.Serializable; +import java.util.Objects; + +/** + * A base class for options. + */ +abstract class AbstractOption implements Serializable { + + private static final long serialVersionUID = -5912727967831484228L; + private final Object value; + private final DnsRpc.Option rpcOption; + + AbstractOption(DnsRpc.Option rpcOption, Object value) { + this.rpcOption = checkNotNull(rpcOption); + this.value = value; + } + + Object value() { + return value; + } + + DnsRpc.Option rpcOption() { + return rpcOption; + } + + @Override + public boolean equals(Object obj) { + if (!(obj instanceof AbstractOption)) { + return false; + } + AbstractOption other = (AbstractOption) obj; + return Objects.equals(value, other.value) && Objects.equals(rpcOption, other.rpcOption); + } + + @Override + public int hashCode() { + return Objects.hash(value, rpcOption); + } + + @Override + public String toString() { + return MoreObjects.toStringHelper(this) + .add("value", value) + .add("rpcOption", rpcOption) + .toString(); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ChangeRequest.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ChangeRequest.java new file mode 100644 index 000000000000..4b6369976ca6 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ChangeRequest.java @@ -0,0 +1,197 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.api.services.dns.model.Change; +import com.google.common.base.Function; + +import java.io.IOException; +import java.io.ObjectInputStream; +import java.util.List; +import java.util.Objects; + +/** + * An immutable class representing an atomic update to a collection of {@link RecordSet}s within a + * {@code Zone}. + * + * @see Google Cloud DNS documentation + */ +public class ChangeRequest extends ChangeRequestInfo { + + private static final long serialVersionUID = 5335667200595081449L; + private final DnsOptions options; + private final String zone; + private transient Dns dns; + + /** + * A builder for {@code ChangeRequest}s. + */ + public static class Builder extends ChangeRequestInfo.Builder { + + private final Dns dns; + private final String zone; + private final ChangeRequestInfo.BuilderImpl infoBuilder; + + private Builder(ChangeRequest cr) { + this.dns = cr.dns; + this.zone = cr.zone; + this.infoBuilder = new ChangeRequestInfo.BuilderImpl(cr); + } + + @Override + public Builder additions(List additions) { + infoBuilder.additions(additions); + return this; + } + + @Override + public Builder deletions(List deletions) { + infoBuilder.deletions(deletions); + return this; + } + + @Override + public Builder add(RecordSet recordSet) { + infoBuilder.add(recordSet); + return this; + } + + @Override + public Builder delete(RecordSet recordSet) { + infoBuilder.delete(recordSet); + return this; + } + + @Override + public Builder clearAdditions() { + infoBuilder.clearAdditions(); + return this; + } + + @Override + public Builder clearDeletions() { + infoBuilder.clearDeletions(); + return this; + } + + @Override + public Builder removeAddition(RecordSet recordSet) { + infoBuilder.removeAddition(recordSet); + return this; + } + + @Override + public Builder removeDeletion(RecordSet recordSet) { + infoBuilder.removeDeletion(recordSet); + return this; + } + + @Override + Builder id(String id) { + infoBuilder.id(id); + return this; + } + + @Override + Builder startTimeMillis(long startTimeMillis) { + infoBuilder.startTimeMillis(startTimeMillis); + return this; + } + + @Override + Builder status(Status status) { + infoBuilder.status(status); + return this; + } + + @Override + public ChangeRequest build() { + return new ChangeRequest(dns, zone, infoBuilder); + } + } + + ChangeRequest(Dns dns, String zone, ChangeRequest.BuilderImpl infoBuilder) { + super(infoBuilder); + this.zone = checkNotNull(zone); + this.dns = checkNotNull(dns); + this.options = dns.options(); + } + + /** + * Returns the name of the {@link Zone} associated with this change request. + */ + public String zone() { + return this.zone; + } + + /** + * Returns the change request's {@code Dns} object used to issue requests. + */ + public Dns dns() { + return dns; + } + + /** + * Applies this change request to the associated zone. + */ + public ChangeRequest applyTo(Dns.ChangeRequestOption... options) { + return dns.applyChangeRequest(zone, this, options); + } + + @Override + public Builder toBuilder() { + return new Builder(this); + } + + @Override + public boolean equals(Object obj) { + if (obj == null || !obj.getClass().equals(ChangeRequest.class)) { + return false; + } else { + ChangeRequest other = (ChangeRequest) obj; + return Objects.equals(options, other.options) + && Objects.equals(zone, other.zone) + && Objects.equals(toPb(), other.toPb()); + } + } + + @Override + public int hashCode() { + return Objects.hash(super.hashCode(), options, zone); + } + + private void readObject(ObjectInputStream input) throws IOException, ClassNotFoundException { + input.defaultReadObject(); + this.dns = options.service(); + } + + static ChangeRequest fromPb(Dns dns, String zoneName, Change pb) { + ChangeRequestInfo info = ChangeRequestInfo.fromPb(pb); + return new ChangeRequest(dns, zoneName, new ChangeRequestInfo.BuilderImpl(info)); + } + + static Function fromPbFunction(final Dns dns, final String zoneName) { + return new Function() { + @Override + public ChangeRequest apply(com.google.api.services.dns.model.Change pb) { + return ChangeRequest.fromPb(dns, zoneName, pb); + } + }; + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ChangeRequestInfo.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ChangeRequestInfo.java new file mode 100644 index 000000000000..b63b4f4a0788 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ChangeRequestInfo.java @@ -0,0 +1,358 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.api.services.dns.model.Change; +import com.google.common.base.Function; +import com.google.common.base.MoreObjects; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Lists; + +import org.joda.time.DateTime; +import org.joda.time.format.ISODateTimeFormat; + +import java.io.Serializable; +import java.util.LinkedList; +import java.util.List; +import java.util.Objects; + +/** + * A class representing an atomic update to a collection of {@link RecordSet}s within a {@code + * Zone}. + * + * @see Google Cloud DNS documentation + */ +public class ChangeRequestInfo implements Serializable { + + static final Function FROM_PB_FUNCTION = + new Function() { + @Override + public ChangeRequestInfo apply(Change pb) { + return ChangeRequestInfo.fromPb(pb); + } + }; + private static final long serialVersionUID = -9027378042756366333L; + private final List additions; + private final List deletions; + private final String id; + private final Long startTimeMillis; + private final ChangeRequestInfo.Status status; + + /** + * This enumerates the possible states of a change request. + * + * @see Google Cloud DNS + * documentation + */ + public enum Status { + PENDING, + DONE + } + + /** + * A builder for {@code ChangeRequestInfo}. + */ + public abstract static class Builder { + + /** + * Sets a collection of {@link RecordSet}s which are to be added to the zone upon executing this + * {@code ChangeRequestInfo}. + */ + public abstract Builder additions(List additions); + + /** + * Sets a collection of {@link RecordSet}s which are to be deleted from the zone upon executing + * this {@code ChangeRequestInfo}. + */ + public abstract Builder deletions(List deletions); + + /** + * Adds a {@link RecordSet} to be added to the zone upon executing this {@code + * ChangeRequestInfo}. + */ + public abstract Builder add(RecordSet recordSet); + + /** + * Adds a {@link RecordSet} to be deleted to the zone upon executing this + * {@code ChangeRequestInfo}. + */ + public abstract Builder delete(RecordSet recordSet); + + /** + * Clears the collection of {@link RecordSet}s which are to be added to the zone upon executing + * this {@code ChangeRequestInfo}. + */ + public abstract Builder clearAdditions(); + + /** + * Clears the collection of {@link RecordSet}s which are to be deleted from the zone upon + * executing this {@code ChangeRequestInfo}. + */ + public abstract Builder clearDeletions(); + + /** + * Removes a single {@link RecordSet} from the collection of records to be + * added to the zone upon executing this {@code ChangeRequestInfo}. + */ + public abstract Builder removeAddition(RecordSet recordSet); + + /** + * Removes a single {@link RecordSet} from the collection of records to be + * deleted from the zone upon executing this {@code ChangeRequestInfo}. + */ + public abstract Builder removeDeletion(RecordSet recordSet); + + /** + * Associates a server-assigned id to this {@code ChangeRequestInfo}. + */ + abstract Builder id(String id); + + /** + * Sets the time when this change request was started by a server. + */ + abstract Builder startTimeMillis(long startTimeMillis); + + /** + * Sets the current status of this {@code ChangeRequest}. + */ + abstract Builder status(ChangeRequest.Status status); + + /** + * Creates a {@code ChangeRequestInfo} instance populated by the values associated with this + * builder. + */ + public abstract ChangeRequestInfo build(); + } + + static class BuilderImpl extends Builder { + private List additions; + private List deletions; + private String id; + private Long startTimeMillis; + private ChangeRequestInfo.Status status; + + BuilderImpl() { + this.additions = new LinkedList<>(); + this.deletions = new LinkedList<>(); + } + + BuilderImpl(ChangeRequestInfo info) { + this.additions = Lists.newLinkedList(info.additions()); + this.deletions = Lists.newLinkedList(info.deletions()); + this.id = info.id(); + this.startTimeMillis = info.startTimeMillis; + this.status = info.status; + } + + @Override + public Builder additions(List additions) { + this.additions = Lists.newLinkedList(checkNotNull(additions)); + return this; + } + + @Override + public Builder deletions(List deletions) { + this.deletions = Lists.newLinkedList(checkNotNull(deletions)); + return this; + } + + @Override + public Builder add(RecordSet recordSet) { + this.additions.add(checkNotNull(recordSet)); + return this; + } + + @Override + public Builder delete(RecordSet recordSet) { + this.deletions.add(checkNotNull(recordSet)); + return this; + } + + @Override + public Builder clearAdditions() { + this.additions.clear(); + return this; + } + + @Override + public Builder clearDeletions() { + this.deletions.clear(); + return this; + } + + @Override + public Builder removeAddition(RecordSet recordSet) { + this.additions.remove(recordSet); + return this; + } + + @Override + public Builder removeDeletion(RecordSet recordSet) { + this.deletions.remove(recordSet); + return this; + } + + @Override + public ChangeRequestInfo build() { + return new ChangeRequestInfo(this); + } + + @Override + Builder id(String id) { + this.id = checkNotNull(id); + return this; + } + + @Override + Builder startTimeMillis(long startTimeMillis) { + this.startTimeMillis = startTimeMillis; + return this; + } + + @Override + Builder status(ChangeRequestInfo.Status status) { + this.status = checkNotNull(status); + return this; + } + } + + ChangeRequestInfo(BuilderImpl builder) { + this.additions = ImmutableList.copyOf(builder.additions); + this.deletions = ImmutableList.copyOf(builder.deletions); + this.id = builder.id; + this.startTimeMillis = builder.startTimeMillis; + this.status = builder.status; + } + + /** + * Returns an empty builder for the {@code ChangeRequestInfo} class. + */ + public static Builder builder() { + return new BuilderImpl(); + } + + /** + * Creates a builder populated with values of this {@code ChangeRequestInfo}. + */ + public Builder toBuilder() { + return new BuilderImpl(this); + } + + /** + * Returns the list of {@link RecordSet}s to be added to the zone upon submitting this change + * request. + */ + public List additions() { + return additions; + } + + /** + * Returns the list of {@link RecordSet}s to be deleted from the zone upon submitting this change + * request. + */ + public List deletions() { + return deletions; + } + + /** + * Returns the id assigned to this {@code ChangeRequest} by the server. + */ + public String id() { + return id; + } + + /** + * Returns the time when this {@code ChangeRequest} was started by the server. + */ + public Long startTimeMillis() { + return startTimeMillis; + } + + /** + * Returns the status of this {@code ChangeRequest}. If the change request has not been applied + * yet, the status is {@code PENDING}. + */ + public ChangeRequestInfo.Status status() { + return status; + } + + Change toPb() { + Change pb = new Change(); + // set id + if (id() != null) { + pb.setId(id()); + } + // set timestamp + if (startTimeMillis() != null) { + pb.setStartTime(ISODateTimeFormat.dateTime().withZoneUTC().print(startTimeMillis())); + } + // set status + if (status() != null) { + pb.setStatus(status().name().toLowerCase()); + } + // set a list of additions + pb.setAdditions(Lists.transform(additions(), RecordSet.TO_PB_FUNCTION)); + // set a list of deletions + pb.setDeletions(Lists.transform(deletions(), RecordSet.TO_PB_FUNCTION)); + return pb; + } + + static ChangeRequestInfo fromPb(Change pb) { + Builder builder = builder(); + if (pb.getId() != null) { + builder.id(pb.getId()); + } + if (pb.getStartTime() != null) { + builder.startTimeMillis(DateTime.parse(pb.getStartTime()).getMillis()); + } + if (pb.getStatus() != null) { + // we are assuming that status indicated in pb is a lower case version of the enum name + builder.status(ChangeRequest.Status.valueOf(pb.getStatus().toUpperCase())); + } + if (pb.getDeletions() != null) { + builder.deletions(Lists.transform(pb.getDeletions(), RecordSet.FROM_PB_FUNCTION)); + } + if (pb.getAdditions() != null) { + builder.additions(Lists.transform(pb.getAdditions(), RecordSet.FROM_PB_FUNCTION)); + } + return builder.build(); + } + + @Override + public boolean equals(Object other) { + return other != null && other.getClass().equals(ChangeRequestInfo.class) + && other instanceof ChangeRequestInfo && toPb().equals(((ChangeRequestInfo) other).toPb()); + } + + @Override + public int hashCode() { + return Objects.hash(additions, deletions, id, startTimeMillis, status); + } + + @Override + public String toString() { + return MoreObjects.toStringHelper(this) + .add("additions", additions) + .add("deletions", deletions) + .add("id", id) + .add("startTimeMillis", startTimeMillis) + .add("status", status) + .toString(); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/Dns.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/Dns.java new file mode 100644 index 000000000000..f2b42f30a9f6 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/Dns.java @@ -0,0 +1,537 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import com.google.common.base.Joiner; +import com.google.common.collect.Sets; +import com.google.gcloud.Page; +import com.google.gcloud.Service; +import com.google.gcloud.dns.spi.DnsRpc; + +import java.io.Serializable; +import java.util.Set; + +/** + * An interface for the Google Cloud DNS service. + * + * @see Google Cloud DNS + */ +public interface Dns extends Service { + + /** + * The fields of a project. + * + *

These values can be used to specify the fields to include in a partial response when calling + * {@link Dns#getProject(ProjectOption...)}. Project ID is always returned, even if not + * specified. + */ + enum ProjectField { + PROJECT_ID("id"), + PROJECT_NUMBER("number"), + QUOTA("quota"); + + private final String selector; + + ProjectField(String selector) { + this.selector = selector; + } + + String selector() { + return selector; + } + + static String selector(ProjectField... fields) { + Set fieldStrings = Sets.newHashSetWithExpectedSize(fields.length + 1); + fieldStrings.add(PROJECT_ID.selector()); + for (ProjectField field : fields) { + fieldStrings.add(field.selector()); + } + return Joiner.on(',').join(fieldStrings); + } + } + + /** + * The fields of a zone. + * + *

These values can be used to specify the fields to include in a partial response when calling + * {@link Dns#getZone(String, ZoneOption...)}. The name is always returned, even if not + * specified. + */ + enum ZoneField { + CREATION_TIME("creationTime"), + DESCRIPTION("description"), + DNS_NAME("dnsName"), + ZONE_ID("id"), + NAME("name"), + NAME_SERVER_SET("nameServerSet"), + NAME_SERVERS("nameServers"); + + private final String selector; + + ZoneField(String selector) { + this.selector = selector; + } + + String selector() { + return selector; + } + + static String selector(ZoneField... fields) { + Set fieldStrings = Sets.newHashSetWithExpectedSize(fields.length + 1); + fieldStrings.add(NAME.selector()); + for (ZoneField field : fields) { + fieldStrings.add(field.selector()); + } + return Joiner.on(',').join(fieldStrings); + } + } + + /** + * The fields of a record set. + * + *

These values can be used to specify the fields to include in a partial response when calling + * {@link Dns#listRecordSets(String, RecordSetListOption...)}. The name and type are always + * returned even if not selected. + */ + enum RecordSetField { + DNS_RECORDS("rrdatas"), + NAME("name"), + TTL("ttl"), + TYPE("type"); + + private final String selector; + + RecordSetField(String selector) { + this.selector = selector; + } + + String selector() { + return selector; + } + + static String selector(RecordSetField... fields) { + Set fieldStrings = Sets.newHashSetWithExpectedSize(fields.length + 1); + fieldStrings.add(NAME.selector()); + fieldStrings.add(TYPE.selector()); + for (RecordSetField field : fields) { + fieldStrings.add(field.selector()); + } + return Joiner.on(',').join(fieldStrings); + } + } + + /** + * The fields of a change request. + * + *

These values can be used to specify the fields to include in a partial response when calling + * {@link Dns#applyChangeRequest(String, ChangeRequestInfo, ChangeRequestOption...)} The ID is always + * returned even if not selected. + */ + enum ChangeRequestField { + ID("id"), + START_TIME("startTime"), + STATUS("status"), + ADDITIONS("additions"), + DELETIONS("deletions"); + + private final String selector; + + ChangeRequestField(String selector) { + this.selector = selector; + } + + String selector() { + return selector; + } + + static String selector(ChangeRequestField... fields) { + Set fieldStrings = Sets.newHashSetWithExpectedSize(fields.length + 1); + fieldStrings.add(ID.selector()); + for (ChangeRequestField field : fields) { + fieldStrings.add(field.selector()); + } + return Joiner.on(',').join(fieldStrings); + } + } + + /** + * The sorting order for listing. + */ + enum SortingOrder { + DESCENDING, ASCENDING; + + public String selector() { + return name().toLowerCase(); + } + } + + /** + * Class for specifying record set listing options. + */ + class RecordSetListOption extends AbstractOption implements Serializable { + + private static final long serialVersionUID = 1009627025381096098L; + + RecordSetListOption(DnsRpc.Option option, Object value) { + super(option, value); + } + + /** + * Returns an option to specify the record set's fields to be returned by the RPC call. + * + *

If this option is not provided all record fields are returned. {@code + * RecordSetField.fields} can be used to specify only the fields of interest. The name of the + * record set in always returned, even if not specified. {@link RecordSetField} provides a list + * of fields that can be used. + */ + public static RecordSetListOption fields(RecordSetField... fields) { + StringBuilder builder = new StringBuilder(); + builder.append("nextPageToken,rrsets(").append(RecordSetField.selector(fields)) + .append(')'); + return new RecordSetListOption(DnsRpc.Option.FIELDS, builder.toString()); + } + + /** + * Returns an option to specify a page token. + * + *

The page token (returned from a previous call to list) indicates from where listing should + * continue. + */ + public static RecordSetListOption pageToken(String pageToken) { + return new RecordSetListOption(DnsRpc.Option.PAGE_TOKEN, pageToken); + } + + /** + * The maximum number of record sets to return per RPC. + * + *

The server can return fewer record sets than requested. When there are more results than + * the page size, the server will return a page token that can be used to fetch other results. + */ + public static RecordSetListOption pageSize(int pageSize) { + return new RecordSetListOption(DnsRpc.Option.PAGE_SIZE, pageSize); + } + + /** + * Restricts the list to only record sets with this fully qualified domain name. + */ + public static RecordSetListOption dnsName(String dnsName) { + return new RecordSetListOption(DnsRpc.Option.NAME, dnsName); + } + + /** + * Restricts the list to return only record sets of this type. If present, {@link + * RecordSetListOption#dnsName(String)} must also be present. + */ + public static RecordSetListOption type(RecordSet.Type type) { + return new RecordSetListOption(DnsRpc.Option.DNS_TYPE, type.name()); + } + } + + /** + * Class for specifying zone field options. + */ + class ZoneOption extends AbstractOption implements Serializable { + + private static final long serialVersionUID = -8065564464895945037L; + + ZoneOption(DnsRpc.Option option, Object value) { + super(option, value); + } + + /** + * Returns an option to specify the zones's fields to be returned by the RPC call. + * + *

If this option is not provided all zone fields are returned. {@code ZoneOption.fields} can + * be used to specify only the fields of interest. Zone ID is always returned, even if not + * specified. {@link ZoneField} provides a list of fields that can be used. + */ + public static ZoneOption fields(ZoneField... fields) { + return new ZoneOption(DnsRpc.Option.FIELDS, ZoneField.selector(fields)); + } + } + + /** + * Class for specifying zone listing options. + */ + class ZoneListOption extends AbstractOption implements Serializable { + + private static final long serialVersionUID = -2830645032124504717L; + + ZoneListOption(DnsRpc.Option option, Object value) { + super(option, value); + } + + /** + * Returns an option to specify the zones's fields to be returned by the RPC call. + * + *

If this option is not provided all zone fields are returned. {@code ZoneOption.fields} can + * be used to specify only the fields of interest. Zone ID is always returned, even if not + * specified. {@link ZoneField} provides a list of fields that can be used. + */ + public static ZoneListOption fields(ZoneField... fields) { + StringBuilder builder = new StringBuilder(); + builder.append("nextPageToken,managedZones(").append(ZoneField.selector(fields)).append(')'); + return new ZoneListOption(DnsRpc.Option.FIELDS, builder.toString()); + } + + /** + * Returns an option to specify a page token. + * + *

The page token (returned from a previous call to list) indicates from where listing should + * continue. + */ + public static ZoneListOption pageToken(String pageToken) { + return new ZoneListOption(DnsRpc.Option.PAGE_TOKEN, pageToken); + } + + /** + * Restricts the list to only zone with this fully qualified domain name. + */ + public static ZoneListOption dnsName(String dnsName) { + StringBuilder builder = new StringBuilder(); + return new ZoneListOption(DnsRpc.Option.DNS_NAME, dnsName); + } + + /** + * The maximum number of zones to return per RPC. + * + *

The server can return fewer zones than requested. When there are more results than the + * page size, the server will return a page token that can be used to fetch other results. + */ + public static ZoneListOption pageSize(int pageSize) { + return new ZoneListOption(DnsRpc.Option.PAGE_SIZE, pageSize); + } + } + + /** + * Class for specifying project options. + */ + class ProjectOption extends AbstractOption implements Serializable { + + private static final long serialVersionUID = 6817937338218847748L; + + ProjectOption(DnsRpc.Option option, Object value) { + super(option, value); + } + + /** + * Returns an option to specify the project's fields to be returned by the RPC call. + * + *

If this option is not provided all project fields are returned. {@code + * ProjectOption.fields} can be used to specify only the fields of interest. Project ID is + * always returned, even if not specified. {@link ProjectField} provides a list of fields that + * can be used. + */ + public static ProjectOption fields(ProjectField... fields) { + return new ProjectOption(DnsRpc.Option.FIELDS, ProjectField.selector(fields)); + } + } + + /** + * Class for specifying change request field options. + */ + class ChangeRequestOption extends AbstractOption implements Serializable { + + private static final long serialVersionUID = 1067273695061077782L; + + ChangeRequestOption(DnsRpc.Option option, Object value) { + super(option, value); + } + + /** + * Returns an option to specify which fields of {@link ChangeRequest} should be returned by the + * service. + * + *

If this option is not provided all change request fields are returned. {@code + * ChangeRequestOption.fields} can be used to specify only the fields of interest. The ID of the + * change request is always returned, even if not specified. {@link ChangeRequestField} provides + * a list of fields that can be used. + */ + public static ChangeRequestOption fields(ChangeRequestField... fields) { + return new ChangeRequestOption( + DnsRpc.Option.FIELDS, + ChangeRequestField.selector(fields) + ); + } + } + + /** + * Class for specifying change request listing options. + */ + class ChangeRequestListOption extends AbstractOption implements Serializable { + + private static final long serialVersionUID = -900209143895376089L; + + ChangeRequestListOption(DnsRpc.Option option, Object value) { + super(option, value); + } + + /** + * Returns an option to specify which fields of{@link ChangeRequest} should be returned by the + * service. + * + *

If this option is not provided all change request fields are returned. {@code + * ChangeRequestOption.fields} can be used to specify only the fields of interest. The ID of the + * change request is always returned, even if not specified. {@link ChangeRequestField} provides + * a list of fields that can be used. + */ + public static ChangeRequestListOption fields(ChangeRequestField... fields) { + StringBuilder builder = new StringBuilder(); + builder.append("nextPageToken,changes(").append(ChangeRequestField.selector(fields)) + .append(')'); + return new ChangeRequestListOption(DnsRpc.Option.FIELDS, builder.toString()); + } + + /** + * Returns an option to specify a page token. + * + *

The page token (returned from a previous call to list) indicates from where listing should + * continue. + */ + public static ChangeRequestListOption pageToken(String pageToken) { + return new ChangeRequestListOption(DnsRpc.Option.PAGE_TOKEN, pageToken); + } + + /** + * The maximum number of change requests to return per RPC. + * + *

The server can return fewer change requests than requested. When there are more results + * than the page size, the server will return a page token that can be used to fetch other + * results. + */ + public static ChangeRequestListOption pageSize(int pageSize) { + return new ChangeRequestListOption(DnsRpc.Option.PAGE_SIZE, pageSize); + } + + /** + * Returns an option to specify whether the the change requests should be listed in ascending + * (most-recent last) or descending (most-recent first) order with respect to when the change + * request was accepted by the server. If this option is not provided, the listing order is + * undefined. + */ + public static ChangeRequestListOption sortOrder(SortingOrder order) { + return new ChangeRequestListOption(DnsRpc.Option.SORTING_ORDER, order.selector()); + } + } + + /** + * Creates a new zone. + * + *

Returns {@link Zone} object representing the new zone's information. In addition to the + * name, dns name and description (supplied by the user within the {@code zoneInfo} parameter), + * the returned object can include the following read-only fields supplied by the server: creation + * time, id, and list of name servers. The returned fields can be optionally restricted by + * specifying {@link ZoneOption}s. + * + * @throws DnsException upon failure + * @see Cloud DNS Managed Zones: + * create + */ + Zone create(ZoneInfo zoneInfo, ZoneOption... options); + + /** + * Returns the zone by the specified zone name. Returns {@code null} if the zone is not found. The + * returned fields can be optionally restricted by specifying {@link ZoneOption}s. + * + * @throws DnsException upon failure + * @see Cloud DNS Managed Zones: + * get + */ + Zone getZone(String zoneName, ZoneOption... options); + + /** + * Lists the zones inside the project. + * + *

This method returns zones in an unspecified order. New zones do not necessarily appear at + * the end of the list. Use {@link ZoneListOption} to restrict the listing to a domain name, set + * page size, and set page token. + * + * @return a page of zones + * @throws DnsException upon failure + * @see Cloud DNS Managed Zones: + * list + */ + Page listZones(ZoneListOption... options); + + /** + * Deletes an existing zone identified by name. Returns {@code true} if the zone was successfully + * deleted and {@code false} otherwise. + * + * @return {@code true} if zone was found and deleted and {@code false} otherwise + * @throws DnsException upon failure + * @see Cloud DNS Managed Zones: + * delete + */ + boolean delete(String zoneName); // delete does not admit any options + + /** + * Lists the record sets in the zone identified by name. + * + *

The fields to be returned, page size and page tokens can be specified using {@link + * RecordSetListOption}s. + * + * @throws DnsException upon failure or if the zone cannot be found + * @see Cloud DNS + * ResourceRecordSets: list + */ + Page listRecordSets(String zoneName, RecordSetListOption... options); + + /** + * Retrieves the information about the current project. The returned fields can be optionally + * restricted by specifying {@link ProjectOption}s. + * + * @throws DnsException upon failure + * @see Cloud DNS Projects: get + */ + ProjectInfo getProject(ProjectOption... fields); + + /** + * Submits a change request for the specified zone. The returned object contains the following + * read-only fields supplied by the server: id, start time and status. time, id, and list of name + * servers. The fields to be returned can be selected by {@link ChangeRequestOption}s. + * + * @return the new {@link ChangeRequest} + * @throws DnsException upon failure if zone is not found + * @see Cloud DNS Changes: create + */ + ChangeRequest applyChangeRequest(String zoneName, ChangeRequestInfo changeRequest, + ChangeRequestOption... options); + + /** + * Retrieves updated information about a change request previously submitted for a zone identified + * by ID. Returns {@code null} if the request cannot be found and throws an exception if the zone + * does not exist. The fields to be returned using can be specified using {@link + * ChangeRequestOption}s. + * + * @throws DnsException upon failure or if the zone cannot be found + * @see Cloud DNS Chages: get + */ + ChangeRequest getChangeRequest(String zoneName, String changeRequestId, + ChangeRequestOption... options); + + /** + * Lists the change requests for the zone identified by name that were submitted to the service. + * + *

The sorting order for changes (based on when they were received by the server), fields to be + * returned, page size and page token can be specified using {@link ChangeRequestListOption}s. + * + * @return A page of change requests + * @throws DnsException upon failure or if the zone cannot be found + * @see Cloud DNS Chages: list + */ + Page listChangeRequests(String zoneName, ChangeRequestListOption... options); +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsException.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsException.java new file mode 100644 index 000000000000..1ecb98a3fdc6 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsException.java @@ -0,0 +1,66 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import com.google.common.collect.ImmutableSet; +import com.google.gcloud.BaseServiceException; +import com.google.gcloud.RetryHelper.RetryHelperException; +import com.google.gcloud.RetryHelper.RetryInterruptedException; + +import java.io.IOException; +import java.util.Set; + +/** + * DNS service exception. + */ +public class DnsException extends BaseServiceException { + + // see: https://cloud.google.com/dns/troubleshooting + private static final Set RETRYABLE_ERRORS = ImmutableSet.of( + new Error(429, null), + new Error(500, null), + new Error(502, null), + new Error(503, null), + new Error(null, "userRateLimitExceeded"), + new Error(null, "rateLimitExceeded")); + private static final long serialVersionUID = 490302380416260252L; + + public DnsException(IOException exception) { + super(exception, true); + } + + private DnsException(int code, String message) { + super(code, message, null, true); + } + + @Override + protected Set retryableErrors() { + return RETRYABLE_ERRORS; + } + + /** + * Translate RetryHelperException to the DnsException that caused the error. This method will + * always throw an exception. + * + * @throws DnsException when {@code ex} was caused by a {@code DnsException} + * @throws RetryInterruptedException when {@code ex} is a {@code RetryInterruptedException} + */ + static DnsException translateAndThrow(RetryHelperException ex) { + BaseServiceException.translateAndPropagateIfPossible(ex); + throw new DnsException(UNKNOWN_CODE, ex.getMessage()); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsFactory.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsFactory.java new file mode 100644 index 000000000000..734652afb24d --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsFactory.java @@ -0,0 +1,25 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import com.google.gcloud.ServiceFactory; + +/** + * An interface for Dns factories. + */ +public interface DnsFactory extends ServiceFactory { +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsImpl.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsImpl.java new file mode 100644 index 000000000000..51ab0bd92720 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsImpl.java @@ -0,0 +1,320 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.gcloud.RetryHelper.RetryHelperException; +import static com.google.gcloud.RetryHelper.runWithRetries; + +import com.google.api.services.dns.model.Change; +import com.google.api.services.dns.model.ManagedZone; +import com.google.api.services.dns.model.ResourceRecordSet; +import com.google.common.base.Function; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Iterables; +import com.google.common.collect.Maps; +import com.google.gcloud.BaseService; +import com.google.gcloud.Page; +import com.google.gcloud.PageImpl; +import com.google.gcloud.RetryHelper; +import com.google.gcloud.dns.spi.DnsRpc; + +import java.util.Map; +import java.util.concurrent.Callable; + +/** + * A default implementation of Dns. + */ +final class DnsImpl extends BaseService implements Dns { + + private final DnsRpc dnsRpc; + + private static class ZonePageFetcher implements PageImpl.NextPageFetcher { + + private static final long serialVersionUID = 2158209410430566961L; + private final Map requestOptions; + private final DnsOptions serviceOptions; + + ZonePageFetcher(DnsOptions serviceOptions, String cursor, + Map optionMap) { + this.requestOptions = + PageImpl.nextRequestOptions(DnsRpc.Option.PAGE_TOKEN, cursor, optionMap); + this.serviceOptions = serviceOptions; + } + + @Override + public Page nextPage() { + return listZones(serviceOptions, requestOptions); + } + } + + private static class ChangeRequestPageFetcher implements PageImpl.NextPageFetcher { + + private static final long serialVersionUID = 4473265130673029139L; + private final String zoneName; + private final Map requestOptions; + private final DnsOptions serviceOptions; + + ChangeRequestPageFetcher(String zoneName, DnsOptions serviceOptions, String cursor, + Map optionMap) { + this.zoneName = zoneName; + this.requestOptions = + PageImpl.nextRequestOptions(DnsRpc.Option.PAGE_TOKEN, cursor, optionMap); + this.serviceOptions = serviceOptions; + } + + @Override + public Page nextPage() { + return listChangeRequests(zoneName, serviceOptions, requestOptions); + } + } + + private static class DnsRecordPageFetcher implements PageImpl.NextPageFetcher { + + private static final long serialVersionUID = -6039369212511530846L; + private final Map requestOptions; + private final DnsOptions serviceOptions; + private final String zoneName; + + DnsRecordPageFetcher(String zoneName, DnsOptions serviceOptions, String cursor, + Map optionMap) { + this.zoneName = zoneName; + this.requestOptions = + PageImpl.nextRequestOptions(DnsRpc.Option.PAGE_TOKEN, cursor, optionMap); + this.serviceOptions = serviceOptions; + } + + @Override + public Page nextPage() { + return listRecordSets(zoneName, serviceOptions, requestOptions); + } + } + + DnsImpl(DnsOptions options) { + super(options); + dnsRpc = options.rpc(); + } + + @Override + public Page listZones(ZoneListOption... options) { + return listZones(options(), optionMap(options)); + } + + private static Page listZones(final DnsOptions serviceOptions, + final Map optionsMap) { + // define transformation function + // this differs from the other list operations since zone is functional and requires dns service + Function pbToZoneFunction = new Function() { + @Override + public Zone apply( + com.google.api.services.dns.model.ManagedZone zonePb) { + return Zone.fromPb(serviceOptions.service(), zonePb); + } + }; + try { + // get a list of managed zones + final DnsRpc rpc = serviceOptions.rpc(); + DnsRpc.ListResult result = + runWithRetries(new Callable>() { + @Override + public DnsRpc.ListResult call() { + return rpc.listZones(optionsMap); + } + }, serviceOptions.retryParams(), EXCEPTION_HANDLER); + String cursor = result.pageToken(); + // transform that list into zone objects + Iterable zones = result.results() == null + ? ImmutableList.of() : Iterables.transform(result.results(), pbToZoneFunction); + return new PageImpl<>(new ZonePageFetcher(serviceOptions, cursor, optionsMap), + cursor, zones); + } catch (RetryHelperException e) { + throw DnsException.translateAndThrow(e); + } + } + + @Override + public Page listChangeRequests(String zoneName, + ChangeRequestListOption... options) { + return listChangeRequests(zoneName, options(), optionMap(options)); + } + + private static Page listChangeRequests(final String zoneName, + final DnsOptions serviceOptions, final Map optionsMap) { + try { + // get a list of changes + final DnsRpc rpc = serviceOptions.rpc(); + DnsRpc.ListResult result = runWithRetries(new Callable>() { + @Override + public DnsRpc.ListResult call() { + return rpc.listChangeRequests(zoneName, optionsMap); + } + }, serviceOptions.retryParams(), EXCEPTION_HANDLER); + String cursor = result.pageToken(); + // transform that list into change request objects + Iterable changes = result.results() == null + ? ImmutableList.of() + : Iterables.transform(result.results(), + ChangeRequest.fromPbFunction(serviceOptions.service(), zoneName)); + return new PageImpl<>(new ChangeRequestPageFetcher(zoneName, serviceOptions, cursor, + optionsMap), cursor, changes); + } catch (RetryHelperException e) { + throw DnsException.translateAndThrow(e); + } + } + + @Override + public Page listRecordSets(String zoneName, RecordSetListOption... options) { + return listRecordSets(zoneName, options(), optionMap(options)); + } + + private static Page listRecordSets(final String zoneName, + final DnsOptions serviceOptions, final Map optionsMap) { + try { + // get a list of record sets + final DnsRpc rpc = serviceOptions.rpc(); + DnsRpc.ListResult result = runWithRetries( + new Callable>() { + @Override + public DnsRpc.ListResult call() { + return rpc.listRecordSets(zoneName, optionsMap); + } + }, serviceOptions.retryParams(), EXCEPTION_HANDLER); + String cursor = result.pageToken(); + // transform that list into record sets + Iterable recordSets = result.results() == null + ? ImmutableList.of() + : Iterables.transform(result.results(), RecordSet.FROM_PB_FUNCTION); + return new PageImpl<>(new DnsRecordPageFetcher(zoneName, serviceOptions, cursor, optionsMap), + cursor, recordSets); + } catch (RetryHelperException e) { + throw DnsException.translateAndThrow(e); + } + } + + @Override + public Zone create(final ZoneInfo zoneInfo, Dns.ZoneOption... options) { + final Map optionsMap = optionMap(options); + try { + com.google.api.services.dns.model.ManagedZone answer = runWithRetries( + new Callable() { + @Override + public com.google.api.services.dns.model.ManagedZone call() { + return dnsRpc.create(zoneInfo.toPb(), optionsMap); + } + }, options().retryParams(), EXCEPTION_HANDLER); + return answer == null ? null : Zone.fromPb(this, answer); + } catch (RetryHelper.RetryHelperException ex) { + throw DnsException.translateAndThrow(ex); + } + } + + @Override + public Zone getZone(final String zoneName, Dns.ZoneOption... options) { + final Map optionsMap = optionMap(options); + try { + com.google.api.services.dns.model.ManagedZone answer = runWithRetries( + new Callable() { + @Override + public com.google.api.services.dns.model.ManagedZone call() { + return dnsRpc.getZone(zoneName, optionsMap); + } + }, options().retryParams(), EXCEPTION_HANDLER); + return answer == null ? null : Zone.fromPb(this, answer); + } catch (RetryHelper.RetryHelperException ex) { + throw DnsException.translateAndThrow(ex); + } + } + + @Override + public boolean delete(final String zoneName) { + try { + return runWithRetries(new Callable() { + @Override + public Boolean call() { + return dnsRpc.deleteZone(zoneName); + } + }, options().retryParams(), EXCEPTION_HANDLER); + } catch (RetryHelper.RetryHelperException ex) { + throw DnsException.translateAndThrow(ex); + } + } + + @Override + public ProjectInfo getProject(Dns.ProjectOption... fields) { + final Map optionsMap = optionMap(fields); + try { + com.google.api.services.dns.model.Project answer = runWithRetries( + new Callable() { + @Override + public com.google.api.services.dns.model.Project call() { + return dnsRpc.getProject(optionsMap); + } + }, options().retryParams(), EXCEPTION_HANDLER); + return answer == null ? null : ProjectInfo.fromPb(answer); // should never be null + } catch (RetryHelper.RetryHelperException ex) { + throw DnsException.translateAndThrow(ex); + } + } + + @Override + public ChangeRequest applyChangeRequest(final String zoneName, + final ChangeRequestInfo changeRequest, ChangeRequestOption... options) { + final Map optionsMap = optionMap(options); + try { + com.google.api.services.dns.model.Change answer = + runWithRetries( + new Callable() { + @Override + public com.google.api.services.dns.model.Change call() { + return dnsRpc.applyChangeRequest(zoneName, changeRequest.toPb(), optionsMap); + } + }, options().retryParams(), EXCEPTION_HANDLER); + return answer == null ? null : ChangeRequest.fromPb(this, zoneName, answer); // not null + } catch (RetryHelper.RetryHelperException ex) { + throw DnsException.translateAndThrow(ex); + } + } + + @Override + public ChangeRequest getChangeRequest(final String zoneName, final String changeRequestId, + Dns.ChangeRequestOption... options) { + final Map optionsMap = optionMap(options); + try { + com.google.api.services.dns.model.Change answer = + runWithRetries( + new Callable() { + @Override + public com.google.api.services.dns.model.Change call() { + return dnsRpc.getChangeRequest(zoneName, changeRequestId, optionsMap); + } + }, options().retryParams(), EXCEPTION_HANDLER); + return answer == null ? null : ChangeRequest.fromPb(this, zoneName, answer); + } catch (RetryHelper.RetryHelperException ex) { + throw DnsException.translateAndThrow(ex); + } + } + + private Map optionMap(AbstractOption... options) { + Map temp = Maps.newEnumMap(DnsRpc.Option.class); + for (AbstractOption option : options) { + Object prev = temp.put(option.rpcOption(), option.value()); + checkArgument(prev == null, "Duplicate option %s", option); + } + return ImmutableMap.copyOf(temp); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsOptions.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsOptions.java new file mode 100644 index 000000000000..541e7a6c6ea7 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/DnsOptions.java @@ -0,0 +1,116 @@ +/* + * Copyright 2015 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import com.google.common.collect.ImmutableSet; +import com.google.gcloud.ServiceOptions; +import com.google.gcloud.dns.spi.DefaultDnsRpc; +import com.google.gcloud.dns.spi.DnsRpc; +import com.google.gcloud.dns.spi.DnsRpcFactory; + +import java.util.Set; + +public class DnsOptions extends ServiceOptions { + + private static final long serialVersionUID = -519128051411747771L; + private static final String GC_DNS_RW = "https://www.googleapis.com/auth/ndev.clouddns.readwrite"; + private static final Set SCOPES = ImmutableSet.of(GC_DNS_RW); + + public static class DefaultDnsFactory implements DnsFactory { + private static final DnsFactory INSTANCE = new DefaultDnsFactory(); + + @Override + public Dns create(DnsOptions options) { + return new DnsImpl(options); + } + } + + public static class DefaultDnsRpcFactory implements DnsRpcFactory { + + private static final DnsRpcFactory INSTANCE = new DefaultDnsRpcFactory(); + + @Override + public DnsRpc create(DnsOptions options) { + return new DefaultDnsRpc(options); + } + } + + public static class Builder extends ServiceOptions.Builder { + + private Builder() { + } + + private Builder(DnsOptions options) { + super(options); + } + + @Override + public DnsOptions build() { + return new DnsOptions(this); + } + } + + private DnsOptions(Builder builder) { + super(DnsFactory.class, DnsRpcFactory.class, builder); + } + + @SuppressWarnings("unchecked") + @Override + protected DnsFactory defaultServiceFactory() { + return DefaultDnsFactory.INSTANCE; + } + + @SuppressWarnings("unchecked") + @Override + protected DnsRpcFactory defaultRpcFactory() { + return DefaultDnsRpcFactory.INSTANCE; + } + + @Override + protected Set scopes() { + return SCOPES; + } + + @SuppressWarnings("unchecked") + @Override + public Builder toBuilder() { + return new Builder(this); + } + + public static Builder builder() { + return new Builder(); + } + + /** + * Creates a default instance of {@code DnsOptions} with the project ID and credentials inferred + * from the environment. + */ + public static DnsOptions defaultInstance() { + return builder().build(); + } + + @Override + public boolean equals(Object obj) { + return obj instanceof DnsOptions && baseEquals((DnsOptions) obj); + } + + @Override + public int hashCode() { + return baseHashCode(); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ProjectInfo.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ProjectInfo.java new file mode 100644 index 000000000000..319f06ad2444 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ProjectInfo.java @@ -0,0 +1,288 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.api.client.repackaged.com.google.common.base.Preconditions.checkNotNull; + +import com.google.common.base.MoreObjects; + +import java.io.Serializable; +import java.math.BigInteger; +import java.util.Objects; + +/** + * The class provides the Google Cloud DNS information associated with this project. A project is a + * top level container for resources including {@code Zone}s. Projects can be created only in the + * APIs console. + * + * @see Google Cloud DNS documentation + */ +public class ProjectInfo implements Serializable { + + private static final long serialVersionUID = 8696578863323485036L; + private final String id; + private final BigInteger number; + private final Quota quota; + + /** + * This class represents quotas assigned to the {@code ProjectInfo}. + * + * @see Google Cloud DNS + * documentation + */ + public static class Quota implements Serializable { + + private static final long serialVersionUID = 6854685970605363639L; + private final int zones; + private final int resourceRecordsPerRrset; + private final int rrsetAdditionsPerChange; + private final int rrsetDeletionsPerChange; + private final int rrsetsPerZone; + private final int totalRrdataSizePerChange; + + /** + * Creates an instance of {@code Quota}. + * + *

This is the only way of creating an instance of {@code Quota}. As the service does not + * allow for specifying options, quota is an "all-or-nothing object" and we do not need a + * builder. + */ + Quota(int zones, + int resourceRecordsPerRrset, + int rrsetAdditionsPerChange, + int rrsetDeletionsPerChange, + int rrsetsPerZone, + int totalRrdataSizePerChange) { + this.zones = zones; + this.resourceRecordsPerRrset = resourceRecordsPerRrset; + this.rrsetAdditionsPerChange = rrsetAdditionsPerChange; + this.rrsetDeletionsPerChange = rrsetDeletionsPerChange; + this.rrsetsPerZone = rrsetsPerZone; + this.totalRrdataSizePerChange = totalRrdataSizePerChange; + } + + /** + * Returns the maximum allowed number of zones in the project. + */ + public int zones() { + return zones; + } + + /** + * Returns the maximum allowed number of records per {@link RecordSet}. + */ + public int resourceRecordsPerRrset() { + return resourceRecordsPerRrset; + } + + /** + * Returns the maximum allowed number of {@link RecordSet}s to add per {@link + * ChangeRequest}. + */ + public int rrsetAdditionsPerChange() { + return rrsetAdditionsPerChange; + } + + /** + * Returns the maximum allowed number of {@link RecordSet}s to delete per {@link + * ChangeRequest}. + */ + public int rrsetDeletionsPerChange() { + return rrsetDeletionsPerChange; + } + + /** + * Returns the maximum allowed number of {@link RecordSet}s per {@link ZoneInfo} in the + * project. + */ + public int rrsetsPerZone() { + return rrsetsPerZone; + } + + /** + * Returns the maximum allowed size for total records in one ChangesRequest in bytes. + */ + public int totalRrdataSizePerChange() { + return totalRrdataSizePerChange; + } + + @Override + public boolean equals(Object other) { + return (other instanceof Quota) && this.toPb().equals(((Quota) other).toPb()); + } + + @Override + public int hashCode() { + return Objects.hash(zones, resourceRecordsPerRrset, rrsetAdditionsPerChange, + rrsetDeletionsPerChange, rrsetsPerZone, totalRrdataSizePerChange); + } + + com.google.api.services.dns.model.Quota toPb() { + com.google.api.services.dns.model.Quota pb = new com.google.api.services.dns.model.Quota(); + pb.setManagedZones(zones); + pb.setResourceRecordsPerRrset(resourceRecordsPerRrset); + pb.setRrsetAdditionsPerChange(rrsetAdditionsPerChange); + pb.setRrsetDeletionsPerChange(rrsetDeletionsPerChange); + pb.setRrsetsPerManagedZone(rrsetsPerZone); + pb.setTotalRrdataSizePerChange(totalRrdataSizePerChange); + return pb; + } + + static Quota fromPb(com.google.api.services.dns.model.Quota pb) { + Quota quota = new Quota(pb.getManagedZones(), + pb.getResourceRecordsPerRrset(), + pb.getRrsetAdditionsPerChange(), + pb.getRrsetDeletionsPerChange(), + pb.getRrsetsPerManagedZone(), + pb.getTotalRrdataSizePerChange() + ); + return quota; + } + + @Override + public String toString() { + return MoreObjects.toStringHelper(this) + .add("zones", zones) + .add("resourceRecordsPerRrset", resourceRecordsPerRrset) + .add("rrsetAdditionsPerChange", rrsetAdditionsPerChange) + .add("rrsetDeletionsPerChange", rrsetDeletionsPerChange) + .add("rrsetsPerZone", rrsetsPerZone) + .add("totalRrdataSizePerChange", totalRrdataSizePerChange) + .toString(); + } + } + + /** + * A builder for {@code ProjectInfo}. + */ + static class Builder { + private String id; + private BigInteger number; + private Quota quota; + + private Builder() { + } + + /** + * Sets an id of the project. + */ + Builder id(String id) { + this.id = checkNotNull(id); + return this; + } + + /** + * Sets a number of the project. + */ + Builder number(BigInteger number) { + this.number = checkNotNull(number); + return this; + } + + /** + * Sets quotas assigned to the project. + */ + Builder quota(Quota quota) { + this.quota = checkNotNull(quota); + return this; + } + + /** + * Builds an instance of the {@code ProjectInfo}. + */ + ProjectInfo build() { + return new ProjectInfo(this); + } + } + + private ProjectInfo(Builder builder) { + this.id = builder.id; + this.number = builder.number; + this.quota = builder.quota; + } + + /** + * Returns a builder for {@code ProjectInfo}. + */ + static Builder builder() { + return new Builder(); + } + + /** + * Returns the {@code Quota} object which contains quotas assigned to this project. + */ + public Quota quota() { + return quota; + } + + /** + * Returns project number. For internal use only. + */ + BigInteger number() { + return number; + } + + /** + * Returns project id. For internal use only. + */ + String id() { + return id; + } + + com.google.api.services.dns.model.Project toPb() { + com.google.api.services.dns.model.Project pb = new com.google.api.services.dns.model.Project(); + pb.setId(id); + pb.setNumber(number); + if (this.quota != null) { + pb.setQuota(quota.toPb()); + } + return pb; + } + + static ProjectInfo fromPb(com.google.api.services.dns.model.Project pb) { + Builder builder = builder(); + if (pb.getId() != null) { + builder.id(pb.getId()); + } + if (pb.getNumber() != null) { + builder.number(pb.getNumber()); + } + if (pb.getQuota() != null) { + builder.quota(Quota.fromPb(pb.getQuota())); + } + return builder.build(); + } + + @Override + public boolean equals(Object other) { + return (other instanceof ProjectInfo) && toPb().equals(((ProjectInfo) other).toPb()); + } + + @Override + public int hashCode() { + return Objects.hash(id, number, quota); + } + + @Override + public String toString() { + return MoreObjects.toStringHelper(this) + .add("id", id) + .add("number", number) + .add("quota", quota) + .toString(); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/RecordSet.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/RecordSet.java new file mode 100644 index 000000000000..dc6d956406c3 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/RecordSet.java @@ -0,0 +1,318 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.api.services.dns.model.ResourceRecordSet; +import com.google.common.base.Function; +import com.google.common.base.MoreObjects; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Lists; +import com.google.common.primitives.Ints; + +import java.io.Serializable; +import java.util.LinkedList; +import java.util.List; +import java.util.Objects; +import java.util.concurrent.TimeUnit; + +/** + * A class that represents a Google Cloud DNS record set. + * + *

A {@code RecordSet} is the unit of data that will be returned by the DNS servers upon a DNS + * request for a specific domain. The {@code RecordSet} holds the current state of the DNS records + * that make up a zone. You can read the records but you cannot modify them directly. Rather, you + * edit the records in a zone by creating a {@link ChangeRequest}. + * + * @see Google Cloud DNS + * documentation + */ +public class RecordSet implements Serializable { + + static final Function FROM_PB_FUNCTION = + new Function() { + @Override + public RecordSet apply(ResourceRecordSet pb) { + return RecordSet.fromPb(pb); + } + }; + static final Function TO_PB_FUNCTION = + new Function() { + @Override + public ResourceRecordSet apply(RecordSet recordSet) { + return recordSet.toPb(); + } + }; + private static final long serialVersionUID = 8148009870800115261L; + private final String name; + private final List rrdatas; + private final Integer ttl; // this is in seconds + private final Type type; + + /** + * Enum for the DNS record types supported by Cloud DNS. + * + *

Google Cloud DNS currently supports records of type A, AAAA, CNAME, MX NAPTR, NS, PTR, SOA, + * SPF, SRV, TXT. + * + * @see Cloud DNS + * supported record types + */ + public enum Type { + /** + * Address record, which is used to map host names to their IPv4 address. + */ + A, + /** + * IPv6 Address record, which is used to map host names to their IPv6 address. + */ + AAAA, + /** + * Canonical name record, which is used to alias names. + */ + CNAME, + /** + * Mail exchange record, which is used in routing requests to mail servers. + */ + MX, + /** + * Naming authority pointer record, defined by RFC3403. + */ + NAPTR, + /** + * Name server record, which delegates a DNS zone to an authoritative server. + */ + NS, + /** + * Pointer record, which is often used for reverse DNS lookups. + */ + PTR, + /** + * Start of authority record, which specifies authoritative information about a DNS zone. + */ + SOA, + /** + * Sender policy framework record, which is used in email validation systems. + */ + SPF, + /** + * Service locator record, which is used by some voice over IP, instant messaging protocols and + * other applications. + */ + SRV, + /** + * Text record, which can contain arbitrary text and can also be used to define machine readable + * data such as security or abuse prevention information. + */ + TXT + } + + /** + * A builder for {@link RecordSet}. + */ + public static class Builder { + + private List rrdatas = new LinkedList<>(); + private String name; + private Integer ttl; + private Type type; + + private Builder(String name, Type type) { + this.name = checkNotNull(name); + this.type = checkNotNull(type); + } + + /** + * Creates a builder and pre-populates attributes with the values from the provided {@code + * RecordSet} instance. + */ + private Builder(RecordSet record) { + this.name = record.name; + this.ttl = record.ttl; + this.type = record.type; + this.rrdatas.addAll(record.rrdatas); + } + + /** + * Adds a record to the record set. The records should be as defined in RFC 1035 (section 5) and + * RFC 1034 (section 3.6.1). Examples of records are available in Google DNS documentation. + * + * @see Google + * DNS documentation . + */ + public Builder addRecord(String record) { + this.rrdatas.add(checkNotNull(record)); + return this; + } + + /** + * Removes a record from the set. An exact match is required. + */ + public Builder removeRecord(String record) { + this.rrdatas.remove(checkNotNull(record)); + return this; + } + + /** + * Removes all the records. + */ + public Builder clearRecords() { + this.rrdatas.clear(); + return this; + } + + /** + * Replaces the current records with the provided list of records. + */ + public Builder records(List records) { + this.rrdatas = Lists.newLinkedList(checkNotNull(records)); + return this; + } + + /** + * Sets the name for this record set. For example, www.example.com. + */ + public Builder name(String name) { + this.name = checkNotNull(name); + return this; + } + + /** + * Sets the time that this record can be cached by resolvers. This number must be non-negative. + * The maximum duration must be equivalent to at most {@link Integer#MAX_VALUE} seconds. + * + * @param duration A non-negative number of time units + * @param unit The unit of the ttl parameter + */ + public Builder ttl(int duration, TimeUnit unit) { + checkArgument(duration >= 0, + "Duration cannot be negative. The supplied value was %s.", duration); + checkNotNull(unit); + // we cannot have long because pb does not support it + long converted = unit.toSeconds(duration); + ttl = Ints.checkedCast(converted); + return this; + } + + /** + * The identifier of a supported record type, for example, A, AAAA, MX, TXT, and so on. + */ + public Builder type(Type type) { + this.type = checkNotNull(type); + return this; + } + + /** + * Builds the record set. + */ + public RecordSet build() { + return new RecordSet(this); + } + } + + private RecordSet(Builder builder) { + this.name = builder.name; + this.rrdatas = ImmutableList.copyOf(builder.rrdatas); + this.ttl = builder.ttl; + this.type = builder.type; + } + + /** + * Creates a builder pre-populated with the attribute values of this instance. + */ + public Builder toBuilder() { + return new Builder(this); + } + + /** + * Creates a {@code RecordSet} builder for the given {@code name} and {@code type}. + */ + public static Builder builder(String name, Type type) { + return new Builder(name, type); + } + + /** + * Returns the user-assigned name of this record set. + */ + public String name() { + return name; + } + + /** + * Returns a list of records stored in this record set. + */ + public List records() { + return rrdatas; + } + + /** + * Returns the number of seconds that this record set can be cached by resolvers. + */ + public Integer ttl() { + return ttl; + } + + /** + * Returns the type of this record set. + */ + public Type type() { + return type; + } + + @Override + public int hashCode() { + return Objects.hash(name, rrdatas, ttl, type); + } + + @Override + public boolean equals(Object obj) { + return obj instanceof RecordSet && Objects.equals(this.toPb(), ((RecordSet) obj).toPb()); + } + + com.google.api.services.dns.model.ResourceRecordSet toPb() { + com.google.api.services.dns.model.ResourceRecordSet pb = + new com.google.api.services.dns.model.ResourceRecordSet(); + pb.setName(this.name()); + pb.setRrdatas(this.records()); + pb.setTtl(this.ttl()); + pb.setType(this.type().name()); + return pb; + } + + static RecordSet fromPb(com.google.api.services.dns.model.ResourceRecordSet pb) { + Builder builder = builder(pb.getName(), Type.valueOf(pb.getType())); + if (pb.getRrdatas() != null) { + builder.records(pb.getRrdatas()); + } + if (pb.getTtl() != null) { + builder.ttl(pb.getTtl(), TimeUnit.SECONDS); + } + return builder.build(); + } + + @Override + public String toString() { + return MoreObjects.toStringHelper(this) + .add("name", name()) + .add("rrdatas", records()) + .add("ttl", ttl()) + .add("type", type()) + .toString(); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/Zone.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/Zone.java new file mode 100644 index 000000000000..9930bfdbad67 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/Zone.java @@ -0,0 +1,217 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.gcloud.Page; + +import java.io.IOException; +import java.io.ObjectInputStream; +import java.util.List; +import java.util.Objects; + +/** + * A Google Cloud DNS Zone object. + * + *

A zone is the container for all of your record sets that share the same DNS name prefix, for + * example, example.com. Zones are automatically assigned a set of name servers when they are + * created to handle responding to DNS queries for that zone. A zone has quotas for the number of + * record sets that it can include. + * + * @see Google Cloud DNS managed zone + * documentation + */ +public class Zone extends ZoneInfo { + + private static final long serialVersionUID = -5817771337847861598L; + private final DnsOptions options; + private transient Dns dns; + + /** + * Builder for {@code Zone}. + */ + public static class Builder extends ZoneInfo.Builder { + private final Dns dns; + private final ZoneInfo.BuilderImpl infoBuilder; + + private Builder(Zone zone) { + this.dns = zone.dns; + this.infoBuilder = new ZoneInfo.BuilderImpl(zone); + } + + @Override + public Builder name(String name) { + infoBuilder.name(name); + return this; + } + + @Override + Builder id(String id) { + infoBuilder.id(id); + return this; + } + + @Override + Builder creationTimeMillis(long creationTimeMillis) { + infoBuilder.creationTimeMillis(creationTimeMillis); + return this; + } + + @Override + public Builder dnsName(String dnsName) { + infoBuilder.dnsName(dnsName); + return this; + } + + @Override + public Builder description(String description) { + infoBuilder.description(description); + return this; + } + + @Override + Builder nameServerSet(String nameServerSet) { + infoBuilder.nameServerSet(nameServerSet); + return this; + } + + @Override + Builder nameServers(List nameServers) { + infoBuilder.nameServers(nameServers); // infoBuilder makes a copy + return this; + } + + @Override + public Zone build() { + return new Zone(dns, infoBuilder); + } + } + + Zone(Dns dns, ZoneInfo.BuilderImpl infoBuilder) { + super(infoBuilder); + this.dns = dns; + this.options = dns.options(); + } + + @Override + public Builder toBuilder() { + return new Builder(this); + } + + /** + * Retrieves the latest information about the zone. The method retrieves the zone by name. + * + * @param options optional restriction on what fields should be fetched + * @return zone object containing updated information or {@code null} if not not found + * @throws DnsException upon failure + */ + public Zone reload(Dns.ZoneOption... options) { + return dns.getZone(name(), options); + } + + /** + * Deletes the zone. The method deletes the zone by name. + * + * @return {@code true} is zone was found and deleted and {@code false} otherwise + * @throws DnsException upon failure + */ + public boolean delete() { + return dns.delete(name()); + } + + /** + * Lists all {@link RecordSet}s associated with this zone. The method searches for zone by name. + * + * @param options optional restriction on listing and fields of {@link RecordSet}s returned + * @return a page of record sets + * @throws DnsException upon failure or if the zone is not found + */ + public Page listRecordSets(Dns.RecordSetListOption... options) { + return dns.listRecordSets(name(), options); + } + + /** + * Submits {@link ChangeRequestInfo} to the service for it to applied to this zone. The method + * searches for zone by name. + * + * @param options optional restriction on what fields of {@link ChangeRequest} should be returned + * @return ChangeRequest with server-assigned ID + * @throws DnsException upon failure or if the zone is not found + */ + public ChangeRequest applyChangeRequest(ChangeRequestInfo changeRequest, + Dns.ChangeRequestOption... options) { + checkNotNull(changeRequest); + return dns.applyChangeRequest(name(), changeRequest, options); + } + + /** + * Retrieves an updated information about a change request previously submitted to be applied to + * this zone. Returns a {@link ChangeRequest} or {@code null} if the change request was not found. + * Throws {@link DnsException} if the zone is not found. + * + * @param options optional restriction on what fields of {@link ChangeRequest} should be returned + * @return updated ChangeRequest + * @throws DnsException upon failure or if the zone is not found + * @throws NullPointerException if {@code changeRequestId} is null + */ + public ChangeRequest getChangeRequest(String changeRequestId, + Dns.ChangeRequestOption... options) { + checkNotNull(changeRequestId); + return dns.getChangeRequest(name(), changeRequestId, options); + } + + /** + * Retrieves all change requests for this zone. The method searches for zone by name. Returns a + * page of {@link ChangeRequest}s. + * + * @param options optional restriction on listing and fields to be returned + * @return a page of change requests + * @throws DnsException upon failure or if the zone is not found + */ + public Page listChangeRequests(Dns.ChangeRequestListOption... options) { + return dns.listChangeRequests(name(), options); + } + + /** + * Returns the {@link Dns} service object associated with this zone. + */ + public Dns dns() { + return this.dns; + } + + @Override + public boolean equals(Object obj) { + return obj instanceof Zone && Objects.equals(toPb(), ((Zone) obj).toPb()) + && Objects.equals(options, ((Zone) obj).options); + } + + @Override + public int hashCode() { + return Objects.hash(super.hashCode(), options); + } + + private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException { + in.defaultReadObject(); + this.dns = options.service(); + } + + static Zone fromPb(Dns dns, com.google.api.services.dns.model.ManagedZone zone) { + ZoneInfo info = ZoneInfo.fromPb(zone); + return new Zone(dns, new ZoneInfo.BuilderImpl(info)); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ZoneInfo.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ZoneInfo.java new file mode 100644 index 000000000000..38a88b67777e --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/ZoneInfo.java @@ -0,0 +1,317 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.common.base.Preconditions.checkNotNull; + +import com.google.common.base.MoreObjects; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Lists; + +import org.joda.time.DateTime; +import org.joda.time.format.ISODateTimeFormat; + +import java.io.Serializable; +import java.math.BigInteger; +import java.util.List; +import java.util.Objects; + +/** + * A {@code Zone} represents a DNS zone hosted by the Google Cloud DNS service. A zone is a subtree + * of the DNS namespace under one administrative responsibility. See Google Cloud DNS documentation for + * more information. + */ +public class ZoneInfo implements Serializable { + + private static final long serialVersionUID = 201601191647L; + private final String name; + private final String id; + private final Long creationTimeMillis; + private final String dnsName; + private final String description; + private final String nameServerSet; + private final List nameServers; + + /** + * Builder for {@code ZoneInfo}. + */ + public abstract static class Builder { + /** + * Sets a mandatory user-provided name for the zone. It must be unique within the project. + */ + public abstract Builder name(String name); + + /** + * Sets an id for the zone which is assigned to the zone by the server. + */ + abstract Builder id(String id); + + /** + * Sets the time when this zone was created. + */ + abstract Builder creationTimeMillis(long creationTimeMillis); + + /** + * Sets a mandatory DNS name of this zone, for instance "example.com.". + */ + public abstract Builder dnsName(String dnsName); + + /** + * Sets a mandatory description for this zone. The value is a string of at most 1024 characters + * which has no effect on the zone's function. + */ + public abstract Builder description(String description); + + /** + * Optionally specifies the NameServerSet for this zone. A NameServerSet is a set of DNS name + * servers that all host the same zones. Most users will not need to specify this value. + */ + abstract Builder nameServerSet(String nameServerSet); + // this should not be included in tooling as per the service owners + + /** + * Sets a list of servers that hold the information about the zone. This information is provided + * by Google Cloud DNS and is read only. + */ + abstract Builder nameServers(List nameServers); + + /** + * Builds the instance of {@code ZoneInfo} based on the information set by this builder. + */ + public abstract ZoneInfo build(); + } + + static class BuilderImpl extends Builder { + private String name; + private String id; + private Long creationTimeMillis; + private String dnsName; + private String description; + private String nameServerSet; + private List nameServers; + + private BuilderImpl(String name) { + this.name = checkNotNull(name); + } + + /** + * Creates a builder from an existing ZoneInfo object. + */ + BuilderImpl(ZoneInfo info) { + this.name = info.name; + this.id = info.id; + this.creationTimeMillis = info.creationTimeMillis; + this.dnsName = info.dnsName; + this.description = info.description; + this.nameServerSet = info.nameServerSet; + if (info.nameServers != null) { + this.nameServers = ImmutableList.copyOf(info.nameServers); + } + } + + @Override + public Builder name(String name) { + this.name = checkNotNull(name); + return this; + } + + @Override + Builder id(String id) { + this.id = id; + return this; + } + + @Override + Builder creationTimeMillis(long creationTimeMillis) { + this.creationTimeMillis = creationTimeMillis; + return this; + } + + @Override + public Builder dnsName(String dnsName) { + this.dnsName = checkNotNull(dnsName); + return this; + } + + @Override + public Builder description(String description) { + this.description = checkNotNull(description); + return this; + } + + @Override + Builder nameServerSet(String nameServerSet) { + this.nameServerSet = checkNotNull(nameServerSet); + return this; + } + + @Override + Builder nameServers(List nameServers) { + checkNotNull(nameServers); + this.nameServers = Lists.newLinkedList(nameServers); + return this; + } + + @Override + public ZoneInfo build() { + return new ZoneInfo(this); + } + } + + ZoneInfo(BuilderImpl builder) { + this.name = builder.name; + this.id = builder.id; + this.creationTimeMillis = builder.creationTimeMillis; + this.dnsName = builder.dnsName; + this.description = builder.description; + this.nameServerSet = builder.nameServerSet; + this.nameServers = builder.nameServers == null + ? null : ImmutableList.copyOf(builder.nameServers); + } + + /** + * Returns a ZoneInfo object with assigned {@code name}, {@code dnsName} and {@code description}. + */ + public static ZoneInfo of(String name, String dnsName, String description) { + return new BuilderImpl(name).dnsName(dnsName).description(description).build(); + } + + /** + * Returns the user-defined name of the zone. + */ + public String name() { + return name; + } + + /** + * Returns the read-only zone id assigned by the server. + */ + public String id() { + return id; + } + + /** + * Returns the time when this zone was created on the server. + */ + public Long creationTimeMillis() { + return creationTimeMillis; + } + + /** + * Returns the DNS name of this zone, for instance "example.com.". + */ + public String dnsName() { + return dnsName; + } + + /** + * Returns the description of this zone. + */ + public String description() { + return description; + } + + /** + * Returns the optionally specified set of DNS name servers that all host this zone. This value is + * set only for specific use cases and is left empty for vast majority of users. + */ + public String nameServerSet() { + return nameServerSet; + } + + /** + * The nameservers that the zone should be delegated to. This is defined by the Google DNS cloud. + */ + public List nameServers() { + return nameServers == null ? ImmutableList.of() : nameServers; + } + + /** + * Returns a builder for {@code ZoneInfo} prepopulated with the metadata of this zone. + */ + public Builder toBuilder() { + return new BuilderImpl(this); + } + + com.google.api.services.dns.model.ManagedZone toPb() { + com.google.api.services.dns.model.ManagedZone pb = + new com.google.api.services.dns.model.ManagedZone(); + pb.setDescription(this.description()); + pb.setDnsName(this.dnsName()); + if (this.id() != null) { + pb.setId(new BigInteger(this.id())); + } + pb.setName(this.name()); + pb.setNameServers(this.nameServers); // do use real attribute value which may be null + pb.setNameServerSet(this.nameServerSet()); + if (this.creationTimeMillis() != null) { + pb.setCreationTime(ISODateTimeFormat.dateTime() + .withZoneUTC() + .print(this.creationTimeMillis())); + } + return pb; + } + + static ZoneInfo fromPb(com.google.api.services.dns.model.ManagedZone pb) { + Builder builder = new BuilderImpl(pb.getName()); + if (pb.getDescription() != null) { + builder.description(pb.getDescription()); + } + if (pb.getDnsName() != null) { + builder.dnsName(pb.getDnsName()); + } + if (pb.getId() != null) { + builder.id(pb.getId().toString()); + } + if (pb.getNameServers() != null) { + builder.nameServers(pb.getNameServers()); + } + if (pb.getNameServerSet() != null) { + builder.nameServerSet(pb.getNameServerSet()); + } + if (pb.getCreationTime() != null) { + builder.creationTimeMillis(DateTime.parse(pb.getCreationTime()).getMillis()); + } + return builder.build(); + } + + @Override + public boolean equals(Object obj) { + return obj != null && obj.getClass().equals(ZoneInfo.class) + && Objects.equals(toPb(), ((ZoneInfo) obj).toPb()); + } + + @Override + public int hashCode() { + return Objects.hash(name, id, creationTimeMillis, dnsName, + description, nameServerSet, nameServers); + } + + @Override + public String toString() { + return MoreObjects.toStringHelper(this) + .add("name", name()) + .add("id", id()) + .add("description", description()) + .add("dnsName", dnsName()) + .add("nameServerSet", nameServerSet()) + .add("nameServers", nameServers()) + .add("creationTimeMillis", creationTimeMillis()) + .toString(); + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/package-info.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/package-info.java new file mode 100644 index 000000000000..36f41852400c --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/package-info.java @@ -0,0 +1,60 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * A client to the Google Cloud DNS. + * + *

Here are two simple usage examples from within Compute/App Engine. + * + * The first snippet shows how to create a zone resource. The complete source code can be found on + * + * CreateAndListZones.java. Note that you need to replace the {@code domainName} with a domain + * name that you own and the ownership of which you verified with Google. + * + *

 {@code
+ * Dns dns = DnsOptions.defaultInstance().service();
+ * String zoneName = "my-unique-zone";
+ * String domainName = "someexampledomain.com.";
+ * String description = "This is a gcloud-java-dns sample zone.";
+ * ZoneInfo zoneInfo = ZoneInfo.of(zoneName, domainName, description);
+ * Zone createdZone = dns.create(zoneInfo);
+ * } 
+ * + *

The second example shows how to create records inside a zone. The complete code can be found + * on + * CreateAndListDnsRecords.java. + * + *

 {@code
+ * Dns dns = DnsOptions.defaultInstance().service();
+ * String zoneName = "my-unique-zone";
+ * Zone zone = dns.getZone(zoneName);
+ * String ip = "12.13.14.15";
+ * RecordSet toCreate = RecordSet.builder("www.someexampledomain.com.", RecordSet.Type.A)
+ *   .ttl(24, TimeUnit.HOURS)
+ *   .addRecord(ip)
+ *   .build();
+ * ChangeRequestInfo changeRequest = ChangeRequestInfo.builder().add(toCreate).build();
+ * zone.applyChangeRequest(changeRequest);
+ * } 
+ * + *

When using gcloud-java from outside of App/Compute Engine, you have to specify a + * project ID and provide + * credentials. + * + * @see Google Cloud DNS + */ +package com.google.gcloud.dns; diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DefaultDnsRpc.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DefaultDnsRpc.java new file mode 100644 index 000000000000..cbebd19d0d73 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DefaultDnsRpc.java @@ -0,0 +1,196 @@ +package com.google.gcloud.dns.spi; + +import static com.google.gcloud.dns.spi.DnsRpc.ListResult.of; +import static com.google.gcloud.dns.spi.DnsRpc.Option.DNS_NAME; +import static com.google.gcloud.dns.spi.DnsRpc.Option.DNS_TYPE; +import static com.google.gcloud.dns.spi.DnsRpc.Option.FIELDS; +import static com.google.gcloud.dns.spi.DnsRpc.Option.NAME; +import static com.google.gcloud.dns.spi.DnsRpc.Option.PAGE_SIZE; +import static com.google.gcloud.dns.spi.DnsRpc.Option.PAGE_TOKEN; +import static com.google.gcloud.dns.spi.DnsRpc.Option.SORTING_ORDER; +import static java.net.HttpURLConnection.HTTP_NOT_FOUND; + +import com.google.api.client.http.HttpRequestInitializer; +import com.google.api.client.http.HttpTransport; +import com.google.api.client.json.jackson.JacksonFactory; +import com.google.api.services.dns.Dns; +import com.google.api.services.dns.model.Change; +import com.google.api.services.dns.model.ChangesListResponse; +import com.google.api.services.dns.model.ManagedZone; +import com.google.api.services.dns.model.ManagedZonesListResponse; +import com.google.api.services.dns.model.Project; +import com.google.api.services.dns.model.ResourceRecordSet; +import com.google.api.services.dns.model.ResourceRecordSetsListResponse; +import com.google.gcloud.dns.DnsException; +import com.google.gcloud.dns.DnsOptions; + +import java.io.IOException; +import java.util.Map; + +/** + * A default implementation of the DnsRpc interface. + */ +public class DefaultDnsRpc implements DnsRpc { + + private static final String SORT_BY = "changeSequence"; + private final Dns dns; + private final DnsOptions options; + + private static DnsException translate(IOException exception) { + return new DnsException(exception); + } + + /** + * Constructs an instance of this rpc client with provided {@link DnsOptions}. + */ + public DefaultDnsRpc(DnsOptions options) { + HttpTransport transport = options.httpTransportFactory().create(); + HttpRequestInitializer initializer = options.httpRequestInitializer(); + this.dns = new Dns.Builder(transport, new JacksonFactory(), initializer) + .setRootUrl(options.host()) + .setApplicationName(options.applicationName()) + .build(); + this.options = options; + } + + @Override + public ManagedZone create(ManagedZone zone, Map options) throws DnsException { + try { + return dns.managedZones() + .create(this.options.projectId(), zone) + .setFields(FIELDS.getString(options)) + .execute(); + } catch (IOException ex) { + throw translate(ex); + } + } + + @Override + public ManagedZone getZone(String zoneName, Map options) throws DnsException { + // just fields option + try { + return dns.managedZones().get(this.options.projectId(), zoneName) + .setFields(FIELDS.getString(options)) + .execute(); + } catch (IOException ex) { + DnsException serviceException = translate(ex); + if (serviceException.code() == HTTP_NOT_FOUND) { + return null; + } + throw serviceException; + } + } + + @Override + public ListResult listZones(Map options) throws DnsException { + // fields, page token, page size + try { + ManagedZonesListResponse zoneList = dns.managedZones().list(this.options.projectId()) + .setFields(FIELDS.getString(options)) + .setMaxResults(PAGE_SIZE.getInt(options)) + .setDnsName(DNS_NAME.getString(options)) + .setPageToken(PAGE_TOKEN.getString(options)) + .execute(); + return of(zoneList.getNextPageToken(), zoneList.getManagedZones()); + } catch (IOException ex) { + throw translate(ex); + } + } + + @Override + public boolean deleteZone(String zoneName) throws DnsException { + try { + dns.managedZones().delete(this.options.projectId(), zoneName).execute(); + return true; + } catch (IOException ex) { + DnsException serviceException = translate(ex); + if (serviceException.code() == HTTP_NOT_FOUND) { + return false; + } + throw serviceException; + } + } + + @Override + public ListResult listRecordSets(String zoneName, Map options) + throws DnsException { + // options are fields, page token, dns name, type + try { + ResourceRecordSetsListResponse response = dns.resourceRecordSets() + .list(this.options.projectId(), zoneName) + .setFields(FIELDS.getString(options)) + .setPageToken(PAGE_TOKEN.getString(options)) + .setMaxResults(PAGE_SIZE.getInt(options)) + .setName(NAME.getString(options)) + .setType(DNS_TYPE.getString(options)) + .execute(); + return of(response.getNextPageToken(), response.getRrsets()); + } catch (IOException ex) { + throw translate(ex); + } + } + + @Override + public Project getProject(Map options) throws DnsException { + try { + return dns.projects().get(this.options.projectId()) + .setFields(FIELDS.getString(options)).execute(); + } catch (IOException ex) { + throw translate(ex); + } + } + + @Override + public Change applyChangeRequest(String zoneName, Change changeRequest, Map options) + throws DnsException { + try { + return dns.changes().create(this.options.projectId(), zoneName, changeRequest) + .setFields(FIELDS.getString(options)) + .execute(); + } catch (IOException ex) { + throw translate(ex); + } + } + + @Override + public Change getChangeRequest(String zoneName, String changeRequestId, Map options) + throws DnsException { + try { + return dns.changes().get(this.options.projectId(), zoneName, changeRequestId) + .setFields(FIELDS.getString(options)) + .execute(); + } catch (IOException ex) { + DnsException serviceException = translate(ex); + if (serviceException.code() == HTTP_NOT_FOUND) { + if ("entity.parameters.changeId".equals(serviceException.location()) + || (serviceException.getMessage() != null + && serviceException.getMessage().contains("parameters.changeId"))) { + // the change id was not found, but the zone exists + return null; + } + // the zone does not exist, so throw an exception + } + throw serviceException; + } + } + + @Override + public ListResult listChangeRequests(String zoneName, Map options) + throws DnsException { + // options are fields, page token, page size, sort order + try { + Dns.Changes.List request = dns.changes().list(this.options.projectId(), zoneName) + .setFields(FIELDS.getString(options)) + .setMaxResults(PAGE_SIZE.getInt(options)) + .setPageToken(PAGE_TOKEN.getString(options)); + if (SORTING_ORDER.getString(options) != null) { + // todo check and change if more sorting options are implemented, issue #604 + request = request.setSortBy(SORT_BY).setSortOrder(SORTING_ORDER.getString(options)); + } + ChangesListResponse response = request.execute(); + return of(response.getNextPageToken(), response.getChanges()); + } catch (IOException ex) { + throw translate(ex); + } + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DnsRpc.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DnsRpc.java new file mode 100644 index 000000000000..c7478016db27 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DnsRpc.java @@ -0,0 +1,174 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns.spi; + +import com.google.api.services.dns.model.Change; +import com.google.api.services.dns.model.ManagedZone; +import com.google.api.services.dns.model.Project; +import com.google.api.services.dns.model.ResourceRecordSet; +import com.google.common.collect.ImmutableList; +import com.google.gcloud.dns.DnsException; + +import java.util.Map; + +public interface DnsRpc { + + enum Option { + FIELDS("fields"), + PAGE_SIZE("maxResults"), + PAGE_TOKEN("pageToken"), + DNS_NAME("dnsName"), + NAME("name"), + DNS_TYPE("type"), + SORTING_ORDER("sortOrder"); + + private final String value; + + Option(String value) { + this.value = value; + } + + public String value() { + return value; + } + + @SuppressWarnings("unchecked") + T get(Map options) { + return (T) options.get(this); + } + + String getString(Map options) { + return get(options); + } + + Integer getInt(Map options) { + return get(options); + } + } + + class ListResult { + + private final Iterable results; + private final String pageToken; + + public ListResult(String pageToken, Iterable results) { + this.results = ImmutableList.copyOf(results); + this.pageToken = pageToken; + } + + public static ListResult of(String pageToken, Iterable list) { + return new ListResult<>(pageToken, list); + } + + public Iterable results() { + return results; + } + + public String pageToken() { + return pageToken; + } + } + + /** + * Creates a new zone. + * + * @param zone a zone to be created + * @param options a map of options for the service call + * @return Updated {@code ManagedZone} object + * @throws DnsException upon failure + */ + ManagedZone create(ManagedZone zone, Map options) throws DnsException; + + /** + * Retrieves and returns an existing zone. + * + * @param zoneName name of the zone to be returned + * @param options a map of options for the service call + * @return a zone or {@code null} if not found + * @throws DnsException upon failure + */ + ManagedZone getZone(String zoneName, Map options) throws DnsException; + + /** + * Lists the zones that exist within the project. + * + * @param options a map of options for the service call + * @throws DnsException upon failure + */ + ListResult listZones(Map options) throws DnsException; + + /** + * Deletes the zone identified by the name. + * + * @return {@code true} if the zone was deleted and {@code false} otherwise + * @throws DnsException upon failure + */ + boolean deleteZone(String zoneName) throws DnsException; + + /** + * Lists record sets for a given zone. + * + * @param zoneName name of the zone to be listed + * @param options a map of options for the service call + * @throws DnsException upon failure or if zone was not found + */ + ListResult listRecordSets(String zoneName, Map options) + throws DnsException; + + /** + * Returns information about the current project. + * + * @param options a map of options for the service call + * @return up-to-date project information + * @throws DnsException upon failure or if the project is not found + */ + Project getProject(Map options) throws DnsException; + + /** + * Applies change request to a zone. + * + * @param zoneName the name of a zone to which the {@code Change} should be applied + * @param changeRequest change to be applied + * @param options a map of options for the service call + * @return updated change object with server-assigned ID + * @throws DnsException upon failure or if zone was not found + */ + Change applyChangeRequest(String zoneName, Change changeRequest, Map options) + throws DnsException; + + /** + * Returns an existing change request. + * + * @param zoneName the name of a zone to which the {@code Change} was be applied + * @param changeRequestId the unique id assigned to the change by the server + * @param options a map of options for the service call + * @return up-to-date change object or {@code null} if change was not found + * @throws DnsException upon failure or if zone was not found + */ + Change getChangeRequest(String zoneName, String changeRequestId, Map options) + throws DnsException; + + /** + * List existing change requests for a zone. + * + * @param zoneName the name of a zone to which the {@code Change}s were be applied + * @param options a map of options for the service call + * @throws DnsException upon failure or if zone was not found + */ + ListResult listChangeRequests(String zoneName, Map options) + throws DnsException; +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DnsRpcFactory.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DnsRpcFactory.java new file mode 100644 index 000000000000..ca1b1a0dd018 --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/spi/DnsRpcFactory.java @@ -0,0 +1,27 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns.spi; + +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.spi.ServiceRpcFactory; + +/** + * An interface for DnsRpc factory. Implementation will be loaded via {@link + * java.util.ServiceLoader}. + */ +public interface DnsRpcFactory extends ServiceRpcFactory { +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/testing/LocalDnsHelper.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/testing/LocalDnsHelper.java new file mode 100644 index 000000000000..0ae2c37b9b4d --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/testing/LocalDnsHelper.java @@ -0,0 +1,1266 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns.testing; + +import static com.google.common.net.InetAddresses.isInetAddress; +import static java.net.HttpURLConnection.HTTP_NO_CONTENT; +import static java.net.HttpURLConnection.HTTP_OK; + +import com.google.api.client.json.JsonFactory; +import com.google.api.client.json.jackson.JacksonFactory; +import com.google.api.services.dns.model.Change; +import com.google.api.services.dns.model.ManagedZone; +import com.google.api.services.dns.model.Project; +import com.google.api.services.dns.model.Quota; +import com.google.api.services.dns.model.ResourceRecordSet; +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Joiner; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.ImmutableSet; +import com.google.common.collect.ImmutableSortedMap; +import com.google.common.collect.Lists; +import com.google.common.collect.Sets; +import com.google.common.io.ByteStreams; +import com.google.gcloud.dns.DnsOptions; + +import com.sun.net.httpserver.Headers; +import com.sun.net.httpserver.HttpExchange; +import com.sun.net.httpserver.HttpHandler; +import com.sun.net.httpserver.HttpServer; + +import org.joda.time.format.ISODateTimeFormat; + +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.math.BigInteger; +import java.net.InetSocketAddress; +import java.net.URI; +import java.net.URISyntaxException; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.NavigableMap; +import java.util.NavigableSet; +import java.util.Random; +import java.util.Set; +import java.util.SortedMap; +import java.util.TreeMap; +import java.util.TreeSet; +import java.util.concurrent.ConcurrentLinkedQueue; +import java.util.concurrent.ConcurrentNavigableMap; +import java.util.concurrent.ConcurrentSkipListMap; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicReference; +import java.util.logging.Level; +import java.util.logging.Logger; +import java.util.regex.Pattern; +import java.util.zip.GZIPInputStream; + +/** + * A local Google Cloud DNS mock. + * + *

The mock runs in a separate thread, listening for HTTP requests on the local machine at an + * ephemeral port. + * + *

While the mock attempts to simulate the service, there are some differences in the behaviour. + * The mock will accept any project ID and never returns a notFound or another error because of + * project ID. It assumes that all project IDs exist and that the user has all the necessary + * privileges to manipulate any project. Similarly, the local simulation does not require + * verification of domain name ownership. Any request for creating a managed zone will be approved. + * The mock does not track quota and will allow the user to exceed it. The mock provides only basic + * validation of the DNS data for record sets of type A and AAAA. It does not validate any other + * record set types. + */ +public class LocalDnsHelper { + + private final ConcurrentSkipListMap projects + = new ConcurrentSkipListMap<>(); + private static final URI BASE_CONTEXT; + private static final Logger log = Logger.getLogger(LocalDnsHelper.class.getName()); + private static final JsonFactory jsonFactory = new JacksonFactory(); + private static final Random ID_GENERATOR = new Random(); + private static final String VERSION = "v1"; + private static final String CONTEXT = "/dns/" + VERSION + "/projects"; + private static final Set ENCODINGS = ImmutableSet.of("gzip", "x-gzip"); + private static final List TYPES = ImmutableList.of("A", "AAAA", "CNAME", "MX", "NAPTR", + "NS", "PTR", "SOA", "SPF", "SRV", "TXT"); + private static final TreeSet FORBIDDEN = Sets.newTreeSet( + ImmutableList.of("google.com.", "com.", "example.com.", "net.", "org.")); + private static final Pattern ZONE_NAME_RE = Pattern.compile("[a-z][a-z0-9-]*"); + private static final ScheduledExecutorService EXECUTORS = + Executors.newScheduledThreadPool(2, Executors.defaultThreadFactory()); + private static final String PROJECT_ID = "dummyprojectid"; + + static { + try { + BASE_CONTEXT = new URI(CONTEXT); + } catch (URISyntaxException e) { + throw new IllegalArgumentException( + "Could not initialize LocalDnsHelper due to URISyntaxException.", e); + } + } + + private long delayChange; + private final HttpServer server; + private final int port; + + /** + * For matching URLs to operations. + */ + private enum CallRegex { + CHANGE_CREATE("POST", CONTEXT + "/[^/]+/managedZones/[^/]+/changes"), + CHANGE_GET("GET", CONTEXT + "/[^/]+/managedZones/[^/]+/changes/[^/]+"), + CHANGE_LIST("GET", CONTEXT + "/[^/]+/managedZones/[^/]+/changes"), + ZONE_CREATE("POST", CONTEXT + "/[^/]+/managedZones"), + ZONE_DELETE("DELETE", CONTEXT + "/[^/]+/managedZones/[^/]+"), + ZONE_GET("GET", CONTEXT + "/[^/]+/managedZones/[^/]+"), + ZONE_LIST("GET", CONTEXT + "/[^/]+/managedZones"), + PROJECT_GET("GET", CONTEXT + "/[^/]+"), + RECORD_LIST("GET", CONTEXT + "/[^/]+/managedZones/[^/]+/rrsets"); + + private String method; + private String pathRegex; + + CallRegex(String method, String pathRegex) { + this.pathRegex = pathRegex; + this.method = method; + } + } + + /** + * Associates a project with a collection of ManagedZones. + */ + static class ProjectContainer { + private final Project project; + private final ConcurrentSkipListMap zones = + new ConcurrentSkipListMap<>(); + + ProjectContainer(Project project) { + this.project = project; + } + + Project project() { + return project; + } + + ConcurrentSkipListMap zones() { + return zones; + } + } + + /** + * Associates a zone with a collection of changes and dns records. + */ + static class ZoneContainer { + private final ManagedZone zone; + private final AtomicReference> + dnsRecords = new AtomicReference<>(ImmutableSortedMap.of()); + private final ConcurrentLinkedQueue changes = new ConcurrentLinkedQueue<>(); + + ZoneContainer(ManagedZone zone) { + this.zone = zone; + this.dnsRecords.set(ImmutableSortedMap.of()); + } + + ManagedZone zone() { + return zone; + } + + AtomicReference> dnsRecords() { + return dnsRecords; + } + + ConcurrentLinkedQueue changes() { + return changes; + } + + Change findChange(String changeId) { + for (Change current : changes) { + if (changeId.equals(current.getId())) { + return current; + } + } + return null; + } + } + + static class Response { + private final int code; + private final String body; + + Response(int code, String body) { + this.code = code; + this.body = body; + } + + int code() { + return code; + } + + String body() { + return body; + } + } + + private enum Error { + REQUIRED(400, "global", "required", "REQUIRED"), + INTERNAL_ERROR(500, "global", "internalError", "INTERNAL_ERROR"), + BAD_REQUEST(400, "global", "badRequest", "BAD_REQUEST"), + INVALID(400, "global", "invalid", "INVALID"), + CONTAINER_NOT_EMPTY(400, "global", "containerNotEmpty", "CONTAINER_NOT_EMPTY"), + NOT_AVAILABLE(400, "global", "managedZoneDnsNameNotAvailable", "NOT_AVAILABLE"), + NOT_FOUND(404, "global", "notFound", "NOT_FOUND"), + ALREADY_EXISTS(409, "global", "alreadyExists", "ALREADY_EXISTS"), + CONDITION_NOT_MET(412, "global", "conditionNotMet", "CONDITION_NOT_MET"), + INVALID_ZONE_APEX(400, "global", "invalidZoneApex", "INVALID_ZONE_APEX"); + + private final int code; + private final String domain; + private final String reason; + private final String status; + + Error(int code, String domain, String reason, String status) { + this.code = code; + this.domain = domain; + this.reason = reason; + this.status = status; + } + + Response response(String message) { + try { + return new Response(code, toJson(message)); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response("Error when generating JSON error response."); + } + } + + private String toJson(String message) throws IOException { + Map errors = new HashMap<>(); + errors.put("domain", domain); + errors.put("message", message); + errors.put("reason", reason); + Map args = new HashMap<>(); + args.put("errors", ImmutableList.of(errors)); + args.put("code", code); + args.put("message", message); + args.put("status", status); + return jsonFactory.toString(ImmutableMap.of("error", args)); + } + } + + private class RequestHandler implements HttpHandler { + + private Response pickHandler(HttpExchange exchange, CallRegex regex) { + URI relative = BASE_CONTEXT.relativize(exchange.getRequestURI()); + String path = relative.getPath(); + String[] tokens = path.split("/"); + String projectId = tokens.length > 0 ? tokens[0] : null; + String zoneName = tokens.length > 2 ? tokens[2] : null; + String changeId = tokens.length > 4 ? tokens[4] : null; + String query = relative.getQuery(); + switch (regex) { + case CHANGE_GET: + return getChange(projectId, zoneName, changeId, query); + case CHANGE_LIST: + return listChanges(projectId, zoneName, query); + case ZONE_GET: + return getZone(projectId, zoneName, query); + case ZONE_DELETE: + return deleteZone(projectId, zoneName); + case ZONE_LIST: + return listZones(projectId, query); + case PROJECT_GET: + return getProject(projectId, query); + case RECORD_LIST: + return listDnsRecords(projectId, zoneName, query); + case ZONE_CREATE: + try { + return handleZoneCreate(exchange, projectId, query); + } catch (IOException ex) { + return Error.BAD_REQUEST.response(ex.getMessage()); + } + case CHANGE_CREATE: + try { + return handleChangeCreate(exchange, projectId, zoneName, query); + } catch (IOException ex) { + return Error.BAD_REQUEST.response(ex.getMessage()); + } + default: + return Error.INTERNAL_ERROR.response("Operation without a handler."); + } + } + + @Override + public void handle(HttpExchange exchange) throws IOException { + String requestMethod = exchange.getRequestMethod(); + String rawPath = exchange.getRequestURI().getRawPath(); + for (CallRegex regex : CallRegex.values()) { + if (requestMethod.equals(regex.method) && rawPath.matches(regex.pathRegex)) { + Response response = pickHandler(exchange, regex); + writeResponse(exchange, response); + return; + } + } + writeResponse(exchange, Error.NOT_FOUND.response(String.format( + "The url %s for %s method does not match any API call.", + requestMethod, exchange.getRequestURI()))); + } + + /** + * @throws IOException if the request cannot be parsed. + */ + private Response handleChangeCreate(HttpExchange exchange, String projectId, String zoneName, + String query) throws IOException { + String requestBody = decodeContent(exchange.getRequestHeaders(), exchange.getRequestBody()); + Change change; + try { + change = jsonFactory.fromString(requestBody, Change.class); + } catch (IllegalArgumentException ex) { + return Error.REQUIRED.response( + "The 'entity.change' parameter is required but was missing."); + } + String[] fields = OptionParsers.parseGetOptions(query); + return createChange(projectId, zoneName, change, fields); + } + + /** + * @throws IOException if the request cannot be parsed. + */ + private Response handleZoneCreate(HttpExchange exchange, String projectId, String query) + throws IOException { + String requestBody = decodeContent(exchange.getRequestHeaders(), exchange.getRequestBody()); + ManagedZone zone; + try { + zone = jsonFactory.fromString(requestBody, ManagedZone.class); + } catch (IllegalArgumentException ex) { + return Error.REQUIRED.response( + "The 'entity.managedZone' parameter is required but was missing."); + } + String[] options = OptionParsers.parseGetOptions(query); + return createZone(projectId, zone, options); + } + } + + private LocalDnsHelper(long delay) { + this.delayChange = delay; + try { + server = HttpServer.create(new InetSocketAddress(0), 0); + port = server.getAddress().getPort(); + server.createContext(CONTEXT, new RequestHandler()); + } catch (IOException e) { + throw new RuntimeException("Could not bind the mock DNS server.", e); + } + } + + /** + * Accessor for testing purposes. + */ + ConcurrentSkipListMap projects() { + return projects; + } + + /** + * Creates new {@link LocalDnsHelper} instance that listens to requests on the local machine. This + * instance processes changes in separate thread. The parameter determines how long a thread + * should wait before processing a change. If it is set to 0, the threading is turned off and the + * mock will behave synchronously. + * + * @param delay delay for processing changes in ms or 0 for synchronous processing + */ + public static LocalDnsHelper create(Long delay) { + return new LocalDnsHelper(delay); + } + + /** + * Returns a {@link DnsOptions} instance that sets the host to use the mock server. + */ + public DnsOptions options() { + return DnsOptions.builder().projectId(PROJECT_ID).host("http://localhost:" + port).build(); + } + + /** + * Starts the thread that runs the local DNS server. + */ + public void start() { + server.start(); + } + + /** + * Stops the thread that runs the mock DNS server. + */ + public void stop() { + server.stop(1); + } + + private static void writeResponse(HttpExchange exchange, Response response) { + exchange.getResponseHeaders().set("Content-type", "application/json; charset=UTF-8"); + OutputStream outputStream = exchange.getResponseBody(); + try { + exchange.getResponseHeaders().add("Connection", "close"); + exchange.sendResponseHeaders(response.code(), response.body().length()); + if (response.code() != 204) { + // the server automatically sends headers and closes output stream when 204 is returned + outputStream.write(response.body().getBytes(StandardCharsets.UTF_8)); + } + outputStream.close(); + } catch (IOException e) { + log.log(Level.WARNING, "IOException encountered when sending response.", e); + } + } + + private static String decodeContent(Headers headers, InputStream inputStream) throws IOException { + List contentEncoding = headers.get("Content-encoding"); + InputStream input = inputStream; + try { + if (contentEncoding != null && !contentEncoding.isEmpty()) { + String encoding = contentEncoding.get(0); + if (ENCODINGS.contains(encoding)) { + input = new GZIPInputStream(inputStream); + } else if (!"identity".equals(encoding)) { + throw new IOException( + "The request has the following unsupported HTTP content encoding: " + encoding); + } + } + return new String(ByteStreams.toByteArray(input), StandardCharsets.UTF_8); + } catch (IOException e) { + throw new IOException("Exception encountered when decoding request content.", e); + } + } + + /** + * Generates a JSON response. + * + * @param context managedZones | projects | rrsets | changes + */ + @VisibleForTesting + static Response toListResponse(List serializedObjects, String context, String pageToken, + boolean includePageToken) { + StringBuilder responseBody = new StringBuilder(); + responseBody.append("{\"").append(context).append("\": ["); + Joiner.on(",").appendTo(responseBody, serializedObjects); + responseBody.append(']'); + // add page token only if it exists and is asked for + if (pageToken != null && includePageToken) { + responseBody.append(",\"nextPageToken\": \"").append(pageToken).append('"'); + } + responseBody.append('}'); + return new Response(HTTP_OK, responseBody.toString()); + } + + /** + * Prepares record sets that are created by default for each zone. + */ + private static ImmutableSortedMap defaultRecords(ManagedZone zone) { + ResourceRecordSet soa = new ResourceRecordSet(); + soa.setTtl(21600); + soa.setName(zone.getDnsName()); + soa.setRrdatas(ImmutableList.of( + // taken from the service + "ns-cloud-c1.googledomains.com. cloud-dns-hostmaster.google.com. 0 21600 3600 1209600 312" + )); + soa.setType("SOA"); + ResourceRecordSet ns = new ResourceRecordSet(); + ns.setTtl(21600); + ns.setName(zone.getDnsName()); + ns.setRrdatas(zone.getNameServers()); + ns.setType("NS"); + String nsId = getUniqueId(ImmutableSet.of()); + String soaId = getUniqueId(ImmutableSet.of(nsId)); + return ImmutableSortedMap.of(nsId, ns, soaId, soa); + } + + /** + * Returns a list of four nameservers randomly chosen from the predefined set. + */ + @VisibleForTesting + static List randomNameservers() { + ArrayList nameservers = Lists.newArrayList( + "dns1.googlecloud.com", "dns2.googlecloud.com", "dns3.googlecloud.com", + "dns4.googlecloud.com", "dns5.googlecloud.com", "dns6.googlecloud.com" + ); + while (nameservers.size() != 4) { + int index = ID_GENERATOR.nextInt(nameservers.size()); + nameservers.remove(index); + } + return nameservers; + } + + /** + * Returns a hex string id (used for a record set) unique within the set of ids. + */ + @VisibleForTesting + static String getUniqueId(Set ids) { + String id; + do { + id = Long.toHexString(System.currentTimeMillis()) + + Long.toHexString(Math.abs(ID_GENERATOR.nextLong())); + } while (ids.contains(id)); + return id; + } + + /** + * Tests if a record set matches name and type (if provided). Used for filtering. + */ + @VisibleForTesting + static boolean matchesCriteria(ResourceRecordSet recordSet, String name, String type) { + if (type != null && !recordSet.getType().equals(type)) { + return false; + } + return name == null || recordSet.getName().equals(name); + } + + /** + * Returns a project container. Never returns {@code null} because we assume that all projects + * exists. + */ + private ProjectContainer findProject(String projectId) { + ProjectContainer defaultProject = createProject(projectId); + projects.putIfAbsent(projectId, defaultProject); + return projects.get(projectId); + } + + /** + * Returns a zone container. Returns {@code null} if zone does not exist within project. + */ + @VisibleForTesting + ZoneContainer findZone(String projectId, String zoneName) { + ProjectContainer projectContainer = findProject(projectId); // never null + return projectContainer.zones().get(zoneName); + } + + /** + * Returns a change found by its id. Returns {@code null} if such a change does not exist. + */ + @VisibleForTesting + Change findChange(String projectId, String zoneName, String changeId) { + ZoneContainer wrapper = findZone(projectId, zoneName); + return wrapper == null ? null : wrapper.findChange(changeId); + } + + /** + * Returns a response to getChange service call. + */ + @VisibleForTesting + Response getChange(String projectId, String zoneName, String changeId, String query) { + Change change = findChange(projectId, zoneName, changeId); + if (change == null) { + ZoneContainer zone = findZone(projectId, zoneName); + if (zone == null) { + return Error.NOT_FOUND.response(String.format( + "The 'parameters.managedZone' resource named '%s' does not exist.", zoneName)); + } + return Error.NOT_FOUND.response(String.format( + "The 'parameters.changeId' resource named '%s' does not exist.", changeId)); + } + String[] fields = OptionParsers.parseGetOptions(query); + Change result = OptionParsers.extractFields(change, fields); + try { + return new Response(HTTP_OK, jsonFactory.toString(result)); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response(String.format( + "Error when serializing change %s in managed zone %s in project %s.", + changeId, zoneName, projectId)); + } + } + + /** + * Returns a response to getZone service call. + */ + @VisibleForTesting + Response getZone(String projectId, String zoneName, String query) { + ZoneContainer container = findZone(projectId, zoneName); + if (container == null) { + return Error.NOT_FOUND.response(String.format( + "The 'parameters.managedZone' resource named '%s' does not exist.", zoneName)); + } + String[] fields = OptionParsers.parseGetOptions(query); + ManagedZone result = OptionParsers.extractFields(container.zone(), fields); + try { + return new Response(HTTP_OK, jsonFactory.toString(result)); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response(String.format( + "Error when serializing managed zone %s in project %s.", zoneName, projectId)); + } + } + + /** + * We assume that every project exists. If we do not have it in the collection yet, we just create + * a new default project instance with default quota. + */ + @VisibleForTesting + Response getProject(String projectId, String query) { + String[] fields = OptionParsers.parseGetOptions(query); + Project project = findProject(projectId).project(); // creates project if needed + Project result = OptionParsers.extractFields(project, fields); + try { + return new Response(HTTP_OK, jsonFactory.toString(result)); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response( + String.format("Error when serializing project %s.", projectId)); + } + } + + /** + * Creates a project. It generates a project number randomly. + */ + private ProjectContainer createProject(String projectId) { + Quota quota = new Quota(); + quota.setManagedZones(10000); + quota.setRrsetsPerManagedZone(10000); + quota.setRrsetAdditionsPerChange(100); + quota.setRrsetDeletionsPerChange(100); + quota.setTotalRrdataSizePerChange(10000); + quota.setResourceRecordsPerRrset(100); + Project project = new Project(); + project.setId(projectId); + project.setNumber(new BigInteger(String.valueOf( + Math.abs(ID_GENERATOR.nextLong() % Long.MAX_VALUE)))); + project.setQuota(quota); + return new ProjectContainer(project); + } + + @VisibleForTesting + Response deleteZone(String projectId, String zoneName) { + ZoneContainer zone = findZone(projectId, zoneName); + ImmutableSortedMap rrsets = zone == null + ? ImmutableSortedMap.of() : zone.dnsRecords().get(); + ImmutableList defaults = ImmutableList.of("NS", "SOA"); + for (ResourceRecordSet current : rrsets.values()) { + if (!defaults.contains(current.getType())) { + return Error.CONTAINER_NOT_EMPTY.response(String.format( + "The resource named '%s' cannot be deleted because it is not empty", zoneName)); + } + } + ProjectContainer projectContainer = projects.get(projectId); + ZoneContainer previous = projectContainer.zones.remove(zoneName); + return previous == null + ? Error.NOT_FOUND.response(String.format( + "The 'parameters.managedZone' resource named '%s' does not exist.", zoneName)) + : new Response(HTTP_NO_CONTENT, "{}"); + } + + /** + * Creates new managed zone and stores it in the collection. Assumes that project exists. + */ + @VisibleForTesting + Response createZone(String projectId, ManagedZone zone, String... fields) { + Response errorResponse = checkZone(zone); + if (errorResponse != null) { + return errorResponse; + } + ManagedZone completeZone = new ManagedZone(); + completeZone.setName(zone.getName()); + completeZone.setDnsName(zone.getDnsName()); + completeZone.setDescription(zone.getDescription()); + completeZone.setNameServerSet(zone.getNameServerSet()); + completeZone.setCreationTime(ISODateTimeFormat.dateTime().withZoneUTC() + .print(System.currentTimeMillis())); + completeZone.setId(BigInteger.valueOf(Math.abs(ID_GENERATOR.nextLong() % Long.MAX_VALUE))); + completeZone.setNameServers(randomNameservers()); + ZoneContainer zoneContainer = new ZoneContainer(completeZone); + ImmutableSortedMap defaultsRecords = defaultRecords(completeZone); + zoneContainer.dnsRecords().set(defaultsRecords); + Change change = new Change(); + change.setAdditions(ImmutableList.copyOf(defaultsRecords.values())); + change.setStatus("done"); + change.setId("0"); + change.setStartTime(ISODateTimeFormat.dateTime().withZoneUTC() + .print(System.currentTimeMillis())); + zoneContainer.changes().add(change); + ProjectContainer projectContainer = findProject(projectId); + ZoneContainer oldValue = projectContainer.zones().putIfAbsent( + completeZone.getName(), zoneContainer); + if (oldValue != null) { + return Error.ALREADY_EXISTS.response(String.format( + "The resource 'entity.managedZone' named '%s' already exists", completeZone.getName())); + } + ManagedZone result = OptionParsers.extractFields(completeZone, fields); + try { + return new Response(HTTP_OK, jsonFactory.toString(result)); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response( + String.format("Error when serializing managed zone %s.", result.getName())); + } + } + + /** + * Creates a new change, stores it, and if delayChange > 0, invokes processing in a new thread. + */ + Response createChange(String projectId, String zoneName, Change change, String... fields) { + ZoneContainer zoneContainer = findZone(projectId, zoneName); + if (zoneContainer == null) { + return Error.NOT_FOUND.response(String.format( + "The 'parameters.managedZone' resource named %s does not exist.", zoneName)); + } + Response response = checkChange(change, zoneContainer); + if (response != null) { + return response; + } + Change completeChange = new Change(); + if (change.getAdditions() != null) { + completeChange.setAdditions(ImmutableList.copyOf(change.getAdditions())); + } + if (change.getDeletions() != null) { + completeChange.setDeletions(ImmutableList.copyOf(change.getDeletions())); + } + /* We need to set ID for the change. We are working in concurrent environment. We know that the + element fell on an index between 1 and maxId (index 0 is the default change which creates SOA + and NS), so we will reset all IDs between 0 and maxId (all of them are valid for the respective + objects). */ + ConcurrentLinkedQueue changeSequence = zoneContainer.changes(); + changeSequence.add(completeChange); + int maxId = changeSequence.size(); + int index = 0; + for (Change c : changeSequence) { + if (index == maxId) { + break; + } + c.setId(String.valueOf(index++)); + } + completeChange.setStatus("pending"); + completeChange.setStartTime(ISODateTimeFormat.dateTime().withZoneUTC() + .print(System.currentTimeMillis())); + invokeChange(projectId, zoneName, completeChange.getId()); + Change result = OptionParsers.extractFields(completeChange, fields); + try { + return new Response(HTTP_OK, jsonFactory.toString(result)); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response( + String.format("Error when serializing change %s in managed zone %s in project %s.", + result.getId(), zoneName, projectId)); + } + } + + /** + * Applies change. Uses a different pooled thread which applies the change only if {@code + * delayChange} is > 0. + */ + private void invokeChange(final String projectId, final String zoneName, + final String changeId) { + if (delayChange > 0) { + EXECUTORS.schedule(new Runnable() { + @Override + public void run() { + applyExistingChange(projectId, zoneName, changeId); + } + }, delayChange, TimeUnit.MILLISECONDS); + } else { + applyExistingChange(projectId, zoneName, changeId); + } + } + + /** + * Applies changes to a zone. Repeatedly tries until succeeds. Thread safe and deadlock safe. + */ + private void applyExistingChange(String projectId, String zoneName, String changeId) { + Change change = findChange(projectId, zoneName, changeId); + if (change == null) { + return; // no such change exists, nothing to do + } + ZoneContainer wrapper = findZone(projectId, zoneName); + if (wrapper == null) { + return; // no such zone exists; it might have been deleted by another thread + } + AtomicReference> dnsRecords = + wrapper.dnsRecords(); + while (true) { + // managed zone must have a set of records which is not null + ImmutableSortedMap original = dnsRecords.get(); + // the copy will be populated when handling deletions + SortedMap copy = new TreeMap<>(); + // apply deletions first + List deletions = change.getDeletions(); + if (deletions != null) { + for (Map.Entry entry : original.entrySet()) { + if (!deletions.contains(entry.getValue())) { + copy.put(entry.getKey(), entry.getValue()); + } + } + } else { + copy.putAll(original); + } + // apply additions + List additions = change.getAdditions(); + if (additions != null) { + for (ResourceRecordSet addition : additions) { + ResourceRecordSet rrset = new ResourceRecordSet(); + rrset.setName(addition.getName()); + rrset.setRrdatas(ImmutableList.copyOf(addition.getRrdatas())); + rrset.setTtl(addition.getTtl()); + rrset.setType(addition.getType()); + String id = getUniqueId(copy.keySet()); + copy.put(id, rrset); + } + } + boolean success = dnsRecords.compareAndSet(original, ImmutableSortedMap.copyOf(copy)); + if (success) { + break; // success if no other thread modified the value in the meantime + } + } + change.setStatus("done"); + } + + /** + * Lists zones. Next page token is the last listed zone name and is returned only of there is more + * to list and if the user does not exclude nextPageToken from field options. + */ + @VisibleForTesting + Response listZones(String projectId, String query) { + Map options = OptionParsers.parseListZonesOptions(query); + Response response = checkListOptions(options); + if (response != null) { + return response; + } + ConcurrentSkipListMap containers = findProject(projectId).zones(); + String[] fields = (String[]) options.get("fields"); + String dnsName = (String) options.get("dnsName"); + String pageToken = (String) options.get("pageToken"); + Integer maxResults = options.get("maxResults") == null + ? null : Integer.valueOf((String) options.get("maxResults")); + boolean sizeReached = false; + boolean hasMorePages = false; + LinkedList serializedZones = new LinkedList<>(); + String lastZoneName = null; + ConcurrentNavigableMap fragment = + pageToken != null ? containers.tailMap(pageToken, false) : containers; + for (ZoneContainer zoneContainer : fragment.values()) { + ManagedZone zone = zoneContainer.zone(); + if (dnsName == null || zone.getDnsName().equals(dnsName)) { + if (sizeReached) { + // we do not add this, just note that there would be more and there should be a token + hasMorePages = true; + break; + } else { + try { + lastZoneName = zone.getName(); + serializedZones.addLast(jsonFactory.toString( + OptionParsers.extractFields(zone, fields))); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response(String.format( + "Error when serializing managed zone %s in project %s", lastZoneName, projectId)); + } + } + } + sizeReached = maxResults != null && maxResults.equals(serializedZones.size()); + } + boolean includePageToken = + hasMorePages && (fields == null || Arrays.asList(fields).contains("nextPageToken")); + return toListResponse(serializedZones, "managedZones", lastZoneName, includePageToken); + } + + /** + * Lists record sets for a zone. Next page token is the ID of the last record listed. + */ + @VisibleForTesting + Response listDnsRecords(String projectId, String zoneName, String query) { + Map options = OptionParsers.parseListDnsRecordsOptions(query); + Response response = checkListOptions(options); + if (response != null) { + return response; + } + ZoneContainer zoneContainer = findZone(projectId, zoneName); + if (zoneContainer == null) { + return Error.NOT_FOUND.response(String.format( + "The 'parameters.managedZone' resource named '%s' does not exist.", zoneName)); + } + ImmutableSortedMap dnsRecords = zoneContainer.dnsRecords().get(); + String[] fields = (String[]) options.get("fields"); + String name = (String) options.get("name"); + String type = (String) options.get("type"); + String pageToken = (String) options.get("pageToken"); + ImmutableSortedMap fragment = + pageToken != null ? dnsRecords.tailMap(pageToken, false) : dnsRecords; + Integer maxResults = options.get("maxResults") == null + ? null : Integer.valueOf((String) options.get("maxResults")); + boolean sizeReached = false; + boolean hasMorePages = false; + LinkedList serializedRrsets = new LinkedList<>(); + String lastRecordId = null; + for (String recordSetId : fragment.keySet()) { + ResourceRecordSet recordSet = fragment.get(recordSetId); + if (matchesCriteria(recordSet, name, type)) { + if (sizeReached) { + // we do not add this, just note that there would be more and there should be a token + hasMorePages = true; + break; + } else { + lastRecordId = recordSetId; + try { + serializedRrsets.addLast(jsonFactory.toString( + OptionParsers.extractFields(recordSet, fields))); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response(String.format( + "Error when serializing resource record set in managed zone %s in project %s", + zoneName, projectId)); + } + } + } + sizeReached = maxResults != null && maxResults.equals(serializedRrsets.size()); + } + boolean includePageToken = + hasMorePages && (fields == null || Arrays.asList(fields).contains("nextPageToken")); + return toListResponse(serializedRrsets, "rrsets", lastRecordId, includePageToken); + } + + /** + * Lists changes. Next page token is the ID of the last change listed. + */ + @VisibleForTesting + Response listChanges(String projectId, String zoneName, String query) { + Map options = OptionParsers.parseListChangesOptions(query); + Response response = checkListOptions(options); + if (response != null) { + return response; + } + ZoneContainer zoneContainer = findZone(projectId, zoneName); + if (zoneContainer == null) { + return Error.NOT_FOUND.response(String.format( + "The 'parameters.managedZone' resource named '%s' does not exist", zoneName)); + } + // take a sorted snapshot of the current change list + NavigableMap changes = new TreeMap<>(); + for (Change c : zoneContainer.changes()) { + if (c.getId() != null) { + changes.put(Integer.valueOf(c.getId()), c); + } + } + String[] fields = (String[]) options.get("fields"); + String sortOrder = (String) options.get("sortOrder"); + String pageToken = (String) options.get("pageToken"); + Integer maxResults = options.get("maxResults") == null + ? null : Integer.valueOf((String) options.get("maxResults")); + // as the only supported field is change sequence, we are not reading sortBy + NavigableSet keys; + if ("descending".equals(sortOrder)) { + keys = changes.descendingKeySet(); + } else { + keys = changes.navigableKeySet(); + } + Integer from = null; + try { + from = Integer.valueOf(pageToken); + } catch (NumberFormatException ex) { + // ignore page token + } + keys = from != null ? keys.tailSet(from, false) : keys; + NavigableMap fragment = + from != null && changes.containsKey(from) ? changes.tailMap(from, false) : changes; + boolean sizeReached = false; + boolean hasMorePages = false; + LinkedList serializedResults = new LinkedList<>(); + String lastChangeId = null; + for (Integer key : keys) { + Change change = fragment.get(key); + if (sizeReached) { + // we do not add this, just note that there would be more and there should be a token + hasMorePages = true; + break; + } else { + lastChangeId = change.getId(); + try { + serializedResults.addLast(jsonFactory.toString( + OptionParsers.extractFields(change, fields))); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response(String.format( + "Error when serializing change %s in managed zone %s in project %s", + lastChangeId, zoneName, projectId)); + } + } + sizeReached = maxResults != null && maxResults.equals(serializedResults.size()); + } + boolean includePageToken = + hasMorePages && (fields == null || Arrays.asList(fields).contains("nextPageToken")); + return toListResponse(serializedResults, "changes", lastChangeId, includePageToken); + } + + /** + * Validates a zone to be created. + */ + private static Response checkZone(ManagedZone zone) { + if (zone.getName() == null) { + return Error.REQUIRED.response( + "The 'entity.managedZone.name' parameter is required but was missing."); + } + if (zone.getDnsName() == null) { + return Error.REQUIRED.response( + "The 'entity.managedZone.dnsName' parameter is required but was missing."); + } + if (zone.getDescription() == null) { + return Error.REQUIRED.response( + "The 'entity.managedZone.description' parameter is required but was missing."); + } + try { + int number = Integer.valueOf(zone.getName()); + return Error.INVALID.response( + String.format("Invalid value for 'entity.managedZone.name': '%s'", number)); + } catch (NumberFormatException ex) { + // expected + } + if (zone.getName().isEmpty() || zone.getName().length() > 32 + || !ZONE_NAME_RE.matcher(zone.getName()).matches()) { + return Error.INVALID.response( + String.format("Invalid value for 'entity.managedZone.name': '%s'", zone.getName())); + } + if (zone.getDnsName().isEmpty() || !zone.getDnsName().endsWith(".")) { + return Error.INVALID.response( + String.format("Invalid value for 'entity.managedZone.dnsName': '%s'", zone.getDnsName())); + } + if (FORBIDDEN.contains(zone.getDnsName())) { + return Error.NOT_AVAILABLE.response(String.format( + "The '%s' managed zone is not available to be created.", zone.getDnsName())); + } + return null; + } + + /** + * Validates a change to be created. + */ + @VisibleForTesting + static Response checkChange(Change change, ZoneContainer zone) { + if ((change.getDeletions() == null || change.getDeletions().size() <= 0) + && (change.getAdditions() == null || change.getAdditions().size() <= 0)) { + return Error.REQUIRED.response("The 'entity.change' parameter is required but was missing."); + } + if (change.getAdditions() != null) { + int counter = 0; + for (ResourceRecordSet addition : change.getAdditions()) { + Response response = checkRrset(addition, zone, counter, "additions"); + if (response != null) { + return response; + } + counter++; + } + } + if (change.getDeletions() != null) { + int counter = 0; + for (ResourceRecordSet deletion : change.getDeletions()) { + Response response = checkRrset(deletion, zone, counter, "deletions"); + if (response != null) { + return response; + } + counter++; + } + } + return checkAdditionsDeletions(change.getAdditions(), change.getDeletions(), zone); + // null if everything is ok + } + + /** + * Checks a rrset within a change. + * + * @param type [additions|deletions] + * @param index the index or addition or deletion in the list + * @param zone the zone that this change is applied to + */ + @VisibleForTesting + static Response checkRrset(ResourceRecordSet rrset, ZoneContainer zone, int index, String type) { + if (rrset.getName() == null || !rrset.getName().endsWith(zone.zone().getDnsName())) { + return Error.INVALID.response(String.format( + "Invalid value for 'entity.change.%s[%s].name': '%s'", type, index, rrset.getName())); + } + if (rrset.getType() == null || !TYPES.contains(rrset.getType())) { + return Error.INVALID.response(String.format( + "Invalid value for 'entity.change.%s[%s].type': '%s'", type, index, rrset.getType())); + } + if (rrset.getTtl() != null && rrset.getTtl() != 0 && rrset.getTtl() < 0) { + return Error.INVALID.response(String.format( + "Invalid value for 'entity.change.%s[%s].ttl': '%s'", type, index, rrset.getTtl())); + } + if (rrset.getRrdatas() == null || rrset.isEmpty()) { + return Error.INVALID.response(String.format( + "Invalid value for 'entity.change.%s[%s].rrdata': '%s'", type, index, "")); + } + int counter = 0; + for (String record : rrset.getRrdatas()) { + if (!checkRrData(record, rrset.getType())) { + return Error.INVALID.response(String.format( + "Invalid value for 'entity.change.%s[%s].rrdata[%s]': '%s'", + type, index, counter, record)); + } + counter++; + } + if ("deletions".equals(type)) { + // check that deletion has a match by name and type + boolean found = false; + for (ResourceRecordSet wrappedRrset : zone.dnsRecords().get().values()) { + if (rrset.getName().equals(wrappedRrset.getName()) + && rrset.getType().equals(wrappedRrset.getType())) { + found = true; + break; + } + } + if (!found) { + return Error.NOT_FOUND.response(String.format( + "The 'entity.change.deletions[%s]' resource named '%s (%s)' does not exist.", + index, rrset.getName(), rrset.getType())); + } + // if found, we still need an exact match + if ("deletions".equals(type) + && !zone.dnsRecords().get().containsValue(rrset)) { + // such a record does not exist + return Error.CONDITION_NOT_MET.response(String.format( + "Precondition not met for 'entity.change.deletions[%s]", index)); + } + } + return null; + } + + /** + * Checks against duplicate additions (for each record set to be added that already exists, we + * must have a matching deletion. Furthermore, check that mandatory SOA and NS records stay. + */ + static Response checkAdditionsDeletions(List additions, + List deletions, ZoneContainer zone) { + if (additions != null) { + int index = 0; + for (ResourceRecordSet rrset : additions) { + for (ResourceRecordSet wrappedRrset : zone.dnsRecords().get().values()) { + if (rrset.getName().equals(wrappedRrset.getName()) + && rrset.getType().equals(wrappedRrset.getType()) + // such a record set exists and we must have a deletion + && (deletions == null || !deletions.contains(wrappedRrset))) { + return Error.ALREADY_EXISTS.response(String.format( + "The 'entity.change.additions[%s]' resource named '%s (%s)' already exists.", + index, rrset.getName(), rrset.getType())); + } + } + if (rrset.getType().equals("SOA") && findByNameAndType(deletions, null, "SOA") == null) { + return Error.INVALID_ZONE_APEX.response(String.format("The resource record set 'entity" + + ".change.additions[%s]' is invalid because a zone must contain exactly one resource" + + " record set of type 'SOA' at the apex.", index)); + } + if (rrset.getType().equals("NS") && findByNameAndType(deletions, null, "NS") == null) { + return Error.INVALID_ZONE_APEX.response(String.format("The resource record set 'entity" + + ".change.additions[%s]' is invalid because a zone must contain exactly one resource" + + " record set of type 'NS' at the apex.", index)); + } + index++; + } + } + if (deletions != null) { + int index = 0; + for (ResourceRecordSet rrset : deletions) { + if (rrset.getType().equals("SOA") && findByNameAndType(additions, null, "SOA") == null) { + return Error.INVALID_ZONE_APEX.response(String.format("The resource record set 'entity" + + ".change.deletions[%s]' is invalid because a zone must contain exactly one resource" + + " record set of type 'SOA' at the apex.", index)); + } + if (rrset.getType().equals("NS") && findByNameAndType(additions, null, "NS") == null) { + return Error.INVALID_ZONE_APEX.response(String.format("The resource record set 'entity" + + ".change.deletions[%s]' is invalid because a zone must contain exactly one resource" + + " record set of type 'NS' at the apex.", index)); + } + index++; + } + } + return null; + } + + /** + * Helper for searching rrsets in a collection. + */ + private static ResourceRecordSet findByNameAndType(Iterable recordSets, + String name, String type) { + if (recordSets != null) { + for (ResourceRecordSet rrset : recordSets) { + if ((name == null || name.equals(rrset.getName())) + && (type == null || type.equals(rrset.getType()))) { + return rrset; + } + } + } + return null; + } + + /** + * We only provide the most basic validation for A and AAAA record sets. + */ + static boolean checkRrData(String data, String type) { + switch (type) { + case "A": + return !data.contains(":") && isInetAddress(data); + case "AAAA": + return data.contains(":") && isInetAddress(data); + default: + return true; + } + } + + /** + * Check supplied listing options. + */ + @VisibleForTesting + static Response checkListOptions(Map options) { + // for general listing + String maxResultsString = (String) options.get("maxResults"); + if (maxResultsString != null) { + Integer maxResults; + try { + maxResults = Integer.valueOf(maxResultsString); + } catch (NumberFormatException ex) { + return Error.INVALID.response(String.format( + "Invalid integer value': '%s'.", maxResultsString)); + } + if (maxResults <= 0) { + return Error.INVALID.response(String.format( + "Invalid value for 'parameters.maxResults': '%s'", maxResults)); + } + } + String dnsName = (String) options.get("dnsName"); + if (dnsName != null && !dnsName.endsWith(".")) { + return Error.INVALID.response(String.format( + "Invalid value for 'parameters.dnsName': '%s'", dnsName)); + } + // for listing record sets, name must be fully qualified + String name = (String) options.get("name"); + if (name != null && !name.endsWith(".")) { + return Error.INVALID.response(String.format( + "Invalid value for 'parameters.name': '%s'", name)); + } + String type = (String) options.get("type"); // must be provided with name + if (type != null) { + if (name == null) { + return Error.INVALID.response("Invalid value for 'parameters.name': '' " + + "(name must be specified if type is specified)"); + } + if (!TYPES.contains(type)) { + return Error.INVALID.response(String.format( + "Invalid value for 'parameters.type': '%s'", type)); + } + } + // for listing changes + String order = (String) options.get("sortOrder"); + if (order != null && !"ascending".equals(order) && !"descending".equals(order)) { + return Error.INVALID.response(String.format( + "Invalid value for 'parameters.sortOrder': '%s'", order)); + } + String sortBy = (String) options.get("sortBy"); // case insensitive + if (sortBy != null && !"changesequence".equals(sortBy.toLowerCase())) { + return Error.INVALID.response(String.format( + "Invalid string value: '%s'. Allowed values: [changesequence]", sortBy)); + } + return null; + } +} diff --git a/gcloud-java-dns/src/main/java/com/google/gcloud/dns/testing/OptionParsers.java b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/testing/OptionParsers.java new file mode 100644 index 000000000000..578a0b52db3d --- /dev/null +++ b/gcloud-java-dns/src/main/java/com/google/gcloud/dns/testing/OptionParsers.java @@ -0,0 +1,255 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns.testing; + +import com.google.api.services.dns.model.Change; +import com.google.api.services.dns.model.ManagedZone; +import com.google.api.services.dns.model.Project; +import com.google.api.services.dns.model.ResourceRecordSet; + +import java.util.HashMap; +import java.util.Map; + +/** + * Utility helpers for LocalDnsHelper. + */ +class OptionParsers { + + static Map parseListZonesOptions(String query) { + Map options = new HashMap<>(); + if (query != null) { + String[] args = query.split("&"); + for (String arg : args) { + String[] argEntry = arg.split("="); + switch (argEntry[0]) { + case "fields": + // List fields are in the form "managedZones(field1, field2, ...),nextPageToken" + String replaced = argEntry[1].replace("managedZones(", ","); + replaced = replaced.replace(")", ","); + // we will get empty strings, but it does not matter, they will be ignored + options.put("fields", replaced.split(",")); + break; + case "dnsName": + options.put("dnsName", argEntry[1]); + break; + case "pageToken": + options.put("pageToken", argEntry[1]); + break; + case "maxResults": + // parsing to int is done while handling + options.put("maxResults", argEntry[1]); + break; + default: + break; + } + } + } + return options; + } + + static String[] parseGetOptions(String query) { + if (query != null) { + String[] args = query.split("&"); + for (String arg : args) { + String[] argEntry = arg.split("="); + if (argEntry[0].equals("fields")) { + // List fields are in the form "fields=field1, field2,..." + return argEntry[1].split(","); + } + } + } + return new String[0]; + } + + static ManagedZone extractFields(ManagedZone fullZone, String... fields) { + if (fields == null || fields.length == 0) { + return fullZone; + } + ManagedZone managedZone = new ManagedZone(); + for (String field : fields) { + switch (field) { + case "creationTime": + managedZone.setCreationTime(fullZone.getCreationTime()); + break; + case "description": + managedZone.setDescription(fullZone.getDescription()); + break; + case "dnsName": + managedZone.setDnsName(fullZone.getDnsName()); + break; + case "id": + managedZone.setId(fullZone.getId()); + break; + case "name": + managedZone.setName(fullZone.getName()); + break; + case "nameServerSet": + managedZone.setNameServerSet(fullZone.getNameServerSet()); + break; + case "nameServers": + managedZone.setNameServers(fullZone.getNameServers()); + break; + default: + break; + } + } + return managedZone; + } + + static Change extractFields(Change fullChange, String... fields) { + if (fields == null || fields.length == 0) { + return fullChange; + } + Change change = new Change(); + for (String field : fields) { + switch (field) { + case "additions": + change.setAdditions(fullChange.getAdditions()); + break; + case "deletions": + change.setDeletions(fullChange.getDeletions()); + break; + case "id": + change.setId(fullChange.getId()); + break; + case "startTime": + change.setStartTime(fullChange.getStartTime()); + break; + case "status": + change.setStatus(fullChange.getStatus()); + break; + default: + break; + } + } + return change; + } + + static Project extractFields(Project fullProject, String... fields) { + if (fields == null || fields.length == 0) { + return fullProject; + } + Project project = new Project(); + for (String field : fields) { + switch (field) { + case "id": + project.setId(fullProject.getId()); + break; + case "number": + project.setNumber(fullProject.getNumber()); + break; + case "quota": + project.setQuota(fullProject.getQuota()); + break; + default: + break; + } + } + return project; + } + + static ResourceRecordSet extractFields(ResourceRecordSet fullRecord, String... fields) { + if (fields == null || fields.length == 0) { + return fullRecord; + } + ResourceRecordSet recordSet = new ResourceRecordSet(); + for (String field : fields) { + switch (field) { + case "name": + recordSet.setName(fullRecord.getName()); + break; + case "rrdatas": + recordSet.setRrdatas(fullRecord.getRrdatas()); + break; + case "type": + recordSet.setType(fullRecord.getType()); + break; + case "ttl": + recordSet.setTtl(fullRecord.getTtl()); + break; + default: + break; + } + } + return recordSet; + } + + static Map parseListChangesOptions(String query) { + Map options = new HashMap<>(); + if (query != null) { + String[] args = query.split("&"); + for (String arg : args) { + String[] argEntry = arg.split("="); + switch (argEntry[0]) { + case "fields": + String replaced = argEntry[1].replace("changes(", ",").replace(")", ","); + options.put("fields", replaced.split(",")); // empty strings will be ignored + break; + case "pageToken": + options.put("pageToken", argEntry[1]); + break; + case "sortBy": + options.put("sortBy", argEntry[1]); + break; + case "sortOrder": + options.put("sortOrder", argEntry[1]); + break; + case "maxResults": + // parsing to int is done while handling + options.put("maxResults", argEntry[1]); + break; + default: + break; + } + } + } + return options; + } + + static Map parseListDnsRecordsOptions(String query) { + Map options = new HashMap<>(); + if (query != null) { + String[] args = query.split("&"); + for (String arg : args) { + String[] argEntry = arg.split("="); + switch (argEntry[0]) { + case "fields": + String replace = argEntry[1].replace("rrsets(", ","); + replace = replace.replace(")", ","); + options.put("fields", replace.split(",")); // empty strings do not matter + break; + case "name": + options.put("name", argEntry[1]); + break; + case "type": + options.put("type", argEntry[1]); + break; + case "pageToken": + options.put("pageToken", argEntry[1]); + break; + case "maxResults": + // parsing to int is done while handling + options.put("maxResults", argEntry[1]); + break; + default: + break; + } + } + } + return options; + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/AbstractOptionTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/AbstractOptionTest.java new file mode 100644 index 000000000000..d88ea85c5846 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/AbstractOptionTest.java @@ -0,0 +1,64 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotEquals; +import static org.junit.Assert.fail; + +import com.google.gcloud.dns.spi.DnsRpc; + +import org.junit.Test; + +public class AbstractOptionTest { + + private static final DnsRpc.Option RPC_OPTION = DnsRpc.Option.DNS_TYPE; + private static final DnsRpc.Option ANOTHER_RPC_OPTION = DnsRpc.Option.DNS_NAME; + private static final String VALUE = "some value"; + private static final String OTHER_VALUE = "another value"; + private static final AbstractOption OPTION = new AbstractOption(RPC_OPTION, VALUE) {}; + private static final AbstractOption OPTION_EQUALS = new AbstractOption(RPC_OPTION, VALUE) {}; + private static final AbstractOption OPTION_NOT_EQUALS1 = + new AbstractOption(RPC_OPTION, OTHER_VALUE) {}; + private static final AbstractOption OPTION_NOT_EQUALS2 = + new AbstractOption(ANOTHER_RPC_OPTION, VALUE) {}; + + @Test + public void testEquals() { + assertEquals(OPTION, OPTION_EQUALS); + assertNotEquals(OPTION, OPTION_NOT_EQUALS1); + assertNotEquals(OPTION, OPTION_NOT_EQUALS2); + } + + @Test + public void testHashCode() { + assertEquals(OPTION.hashCode(), OPTION_EQUALS.hashCode()); + } + + @Test + public void testConstructor() { + assertEquals(RPC_OPTION, OPTION.rpcOption()); + assertEquals(VALUE, OPTION.value()); + try { + new AbstractOption(null, VALUE) {}; + fail("Cannot build with empty option."); + } catch (NullPointerException e) { + // expected + } + new AbstractOption(RPC_OPTION, null) {}; // null value is ok + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ChangeRequestInfoTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ChangeRequestInfoTest.java new file mode 100644 index 000000000000..55f2af0824ec --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ChangeRequestInfoTest.java @@ -0,0 +1,218 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import com.google.common.collect.ImmutableList; + +import org.junit.Test; + +import java.util.List; + +public class ChangeRequestInfoTest { + + private static final String ID = "cr-id-1"; + private static final Long START_TIME_MILLIS = 12334567890L; + private static final ChangeRequest.Status STATUS = ChangeRequest.Status.PENDING; + private static final String NAME1 = "dns1"; + private static final RecordSet.Type TYPE1 = RecordSet.Type.A; + private static final String NAME2 = "dns2"; + private static final RecordSet.Type TYPE2 = RecordSet.Type.AAAA; + private static final String NAME3 = "dns3"; + private static final RecordSet.Type TYPE3 = RecordSet.Type.MX; + private static final RecordSet RECORD1 = RecordSet.builder(NAME1, TYPE1).build(); + private static final RecordSet RECORD2 = RecordSet.builder(NAME2, TYPE2).build(); + private static final RecordSet RECORD3 = RecordSet.builder(NAME3, TYPE3).build(); + private static final List ADDITIONS = ImmutableList.of(RECORD1, RECORD2); + private static final List DELETIONS = ImmutableList.of(RECORD3); + private static final ChangeRequestInfo CHANGE = ChangeRequest.builder() + .add(RECORD1) + .add(RECORD2) + .delete(RECORD3) + .startTimeMillis(START_TIME_MILLIS) + .status(STATUS) + .id(ID) + .build(); + + @Test + public void testEmptyBuilder() { + ChangeRequestInfo cr = ChangeRequest.builder().build(); + assertNotNull(cr.deletions()); + assertTrue(cr.deletions().isEmpty()); + assertNotNull(cr.additions()); + assertTrue(cr.additions().isEmpty()); + } + + @Test + public void testBuilder() { + assertEquals(ID, CHANGE.id()); + assertEquals(STATUS, CHANGE.status()); + assertEquals(START_TIME_MILLIS, CHANGE.startTimeMillis()); + assertEquals(ADDITIONS, CHANGE.additions()); + assertEquals(DELETIONS, CHANGE.deletions()); + List recordList = ImmutableList.of(RECORD1); + ChangeRequestInfo another = CHANGE.toBuilder().additions(recordList).build(); + assertEquals(recordList, another.additions()); + assertEquals(CHANGE.deletions(), another.deletions()); + another = CHANGE.toBuilder().deletions(recordList).build(); + assertEquals(recordList, another.deletions()); + assertEquals(CHANGE.additions(), another.additions()); + } + + @Test + public void testEqualsAndNotEquals() { + ChangeRequestInfo clone = CHANGE.toBuilder().build(); + assertEquals(CHANGE, clone); + clone = ChangeRequest.fromPb(CHANGE.toPb()); + assertEquals(CHANGE, clone); + clone = CHANGE.toBuilder().id("some-other-id").build(); + assertNotEquals(CHANGE, clone); + clone = CHANGE.toBuilder().startTimeMillis(CHANGE.startTimeMillis() + 1).build(); + assertNotEquals(CHANGE, clone); + clone = CHANGE.toBuilder().add(RECORD3).build(); + assertNotEquals(CHANGE, clone); + clone = CHANGE.toBuilder().delete(RECORD1).build(); + assertNotEquals(CHANGE, clone); + ChangeRequestInfo empty = ChangeRequest.builder().build(); + assertNotEquals(CHANGE, empty); + assertEquals(empty, ChangeRequest.builder().build()); + } + + @Test + public void testSameHashCodeOnEquals() { + ChangeRequestInfo clone = CHANGE.toBuilder().build(); + assertEquals(CHANGE, clone); + assertEquals(CHANGE.hashCode(), clone.hashCode()); + ChangeRequestInfo empty = ChangeRequest.builder().build(); + assertEquals(empty.hashCode(), ChangeRequest.builder().build().hashCode()); + } + + @Test + public void testToAndFromPb() { + assertEquals(CHANGE, ChangeRequest.fromPb(CHANGE.toPb())); + ChangeRequestInfo partial = ChangeRequest.builder().build(); + assertEquals(partial, ChangeRequest.fromPb(partial.toPb())); + partial = ChangeRequest.builder().id(ID).build(); + assertEquals(partial, ChangeRequest.fromPb(partial.toPb())); + partial = ChangeRequest.builder().add(RECORD1).build(); + assertEquals(partial, ChangeRequest.fromPb(partial.toPb())); + partial = ChangeRequest.builder().delete(RECORD1).build(); + assertEquals(partial, ChangeRequest.fromPb(partial.toPb())); + partial = ChangeRequest.builder().additions(ADDITIONS).build(); + assertEquals(partial, ChangeRequest.fromPb(partial.toPb())); + partial = ChangeRequest.builder().deletions(DELETIONS).build(); + assertEquals(partial, ChangeRequest.fromPb(partial.toPb())); + partial = ChangeRequest.builder().startTimeMillis(START_TIME_MILLIS).build(); + assertEquals(partial, ChangeRequest.fromPb(partial.toPb())); + partial = ChangeRequest.builder().status(STATUS).build(); + assertEquals(partial, ChangeRequest.fromPb(partial.toPb())); + } + + @Test + public void testToBuilder() { + assertEquals(CHANGE, CHANGE.toBuilder().build()); + ChangeRequestInfo partial = ChangeRequest.builder().build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ChangeRequest.builder().id(ID).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ChangeRequest.builder().add(RECORD1).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ChangeRequest.builder().delete(RECORD1).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ChangeRequest.builder().additions(ADDITIONS).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ChangeRequest.builder().deletions(DELETIONS).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ChangeRequest.builder().startTimeMillis(START_TIME_MILLIS).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ChangeRequest.builder().status(STATUS).build(); + assertEquals(partial, partial.toBuilder().build()); + } + + @Test + public void testClearAdditions() { + ChangeRequestInfo clone = CHANGE.toBuilder().clearAdditions().build(); + assertTrue(clone.additions().isEmpty()); + assertFalse(clone.deletions().isEmpty()); + } + + @Test + public void testAddAddition() { + try { + CHANGE.toBuilder().add(null); + fail("Should not be able to add null RecordSet."); + } catch (NullPointerException e) { + // expected + } + ChangeRequestInfo clone = CHANGE.toBuilder().add(RECORD1).build(); + assertEquals(CHANGE.additions().size() + 1, clone.additions().size()); + } + + @Test + public void testAddDeletion() { + try { + CHANGE.toBuilder().delete(null); + fail("Should not be able to delete null RecordSet."); + } catch (NullPointerException e) { + // expected + } + ChangeRequestInfo clone = CHANGE.toBuilder().delete(RECORD1).build(); + assertEquals(CHANGE.deletions().size() + 1, clone.deletions().size()); + } + + @Test + public void testClearDeletions() { + ChangeRequestInfo clone = CHANGE.toBuilder().clearDeletions().build(); + assertTrue(clone.deletions().isEmpty()); + assertFalse(clone.additions().isEmpty()); + } + + @Test + public void testRemoveAddition() { + ChangeRequestInfo clone = CHANGE.toBuilder().removeAddition(RECORD1).build(); + assertTrue(clone.additions().contains(RECORD2)); + assertFalse(clone.additions().contains(RECORD1)); + assertTrue(clone.deletions().contains(RECORD3)); + clone = CHANGE.toBuilder().removeAddition(RECORD2).removeAddition(RECORD1).build(); + assertFalse(clone.additions().contains(RECORD2)); + assertFalse(clone.additions().contains(RECORD1)); + assertTrue(clone.additions().isEmpty()); + assertTrue(clone.deletions().contains(RECORD3)); + } + + @Test + public void testRemoveDeletion() { + ChangeRequestInfo clone = CHANGE.toBuilder().removeDeletion(RECORD3).build(); + assertTrue(clone.deletions().isEmpty()); + } + + @Test + public void testDateParsing() { + String startTime = "2016-01-26T18:33:43.512Z"; // obtained from service + com.google.api.services.dns.model.Change change = CHANGE.toPb().setStartTime(startTime); + ChangeRequestInfo converted = ChangeRequest.fromPb(change); + assertNotNull(converted.startTimeMillis()); + assertEquals(change, converted.toPb()); + assertEquals(change.getStartTime(), converted.toPb().getStartTime()); + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ChangeRequestTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ChangeRequestTest.java new file mode 100644 index 000000000000..bfd1d0f512f4 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ChangeRequestTest.java @@ -0,0 +1,140 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static org.easymock.EasyMock.createStrictMock; +import static org.easymock.EasyMock.expect; +import static org.easymock.EasyMock.replay; +import static org.easymock.EasyMock.reset; +import static org.easymock.EasyMock.verify; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertSame; +import static org.junit.Assert.assertTrue; + +import com.google.common.collect.ImmutableList; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +public class ChangeRequestTest { + + private static final String ZONE_NAME = "dns-zone-name"; + private static final ChangeRequestInfo CHANGE_REQUEST_INFO = ChangeRequest.builder() + .add(RecordSet.builder("name", RecordSet.Type.A).build()) + .delete(RecordSet.builder("othername", RecordSet.Type.AAAA).build()) + .build(); + private static final DnsOptions OPTIONS = createStrictMock(DnsOptions.class); + + private Dns dns; + private ChangeRequest changeRequest; + private ChangeRequest changeRequestPartial; + + @Before + public void setUp() throws Exception { + dns = createStrictMock(Dns.class); + expect(dns.options()).andReturn(OPTIONS).times(2); + replay(dns); + changeRequest = new ChangeRequest(dns, ZONE_NAME, new ChangeRequestInfo.BuilderImpl( + CHANGE_REQUEST_INFO.toBuilder() + .startTimeMillis(132L) + .id("12") + .status(ChangeRequest.Status.DONE) + .build())); + changeRequestPartial = new ChangeRequest(dns, ZONE_NAME, + new ChangeRequest.BuilderImpl(CHANGE_REQUEST_INFO)); + reset(dns); + } + + @After + public void tearDown() throws Exception { + verify(dns); + } + + @Test + public void testConstructor() { + expect(dns.options()).andReturn(OPTIONS); + replay(dns); + assertEquals(new ChangeRequest(dns, ZONE_NAME, + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_INFO)), changeRequestPartial); + assertNotNull(changeRequest.dns()); + assertEquals(ZONE_NAME, changeRequest.zone()); + assertSame(dns, changeRequestPartial.dns()); + assertEquals(ZONE_NAME, changeRequestPartial.zone()); + } + + @Test + public void testFromPb() { + expect(dns.options()).andReturn(OPTIONS).times(2); + replay(dns); + assertEquals(changeRequest, ChangeRequest.fromPb(dns, ZONE_NAME, changeRequest.toPb())); + assertEquals(changeRequestPartial, + ChangeRequest.fromPb(dns, ZONE_NAME, changeRequestPartial.toPb())); + } + + @Test + public void testEqualsAndToBuilder() { + expect(dns.options()).andReturn(OPTIONS).times(2); + replay(dns); + ChangeRequest compare = changeRequest.toBuilder().build(); + assertEquals(changeRequest, compare); + assertEquals(changeRequest.hashCode(), compare.hashCode()); + compare = changeRequestPartial.toBuilder().build(); + assertEquals(changeRequestPartial, compare); + assertEquals(changeRequestPartial.hashCode(), compare.hashCode()); + } + + @Test + public void testBuilder() { + // one for each build() call because it invokes a constructor + expect(dns.options()).andReturn(OPTIONS).times(9); + replay(dns); + String id = changeRequest.id() + "aaa"; + assertEquals(id, changeRequest.toBuilder().id(id).build().id()); + ChangeRequest modified = + changeRequest.toBuilder().status(ChangeRequest.Status.PENDING).build(); + assertEquals(ChangeRequest.Status.PENDING, modified.status()); + modified = changeRequest.toBuilder().clearDeletions().build(); + assertTrue(modified.deletions().isEmpty()); + modified = changeRequest.toBuilder().clearAdditions().build(); + assertTrue(modified.additions().isEmpty()); + modified = changeRequest.toBuilder().additions(ImmutableList.of()).build(); + assertTrue(modified.additions().isEmpty()); + modified = changeRequest.toBuilder().deletions(ImmutableList.of()).build(); + assertTrue(modified.deletions().isEmpty()); + RecordSet cname = RecordSet.builder("last", RecordSet.Type.CNAME).build(); + modified = changeRequest.toBuilder().add(cname).build(); + assertTrue(modified.additions().contains(cname)); + modified = changeRequest.toBuilder().delete(cname).build(); + assertTrue(modified.deletions().contains(cname)); + modified = changeRequest.toBuilder().startTimeMillis(0L).build(); + assertEquals(Long.valueOf(0), modified.startTimeMillis()); + } + + @Test + public void testApplyTo() { + expect(dns.applyChangeRequest(ZONE_NAME, changeRequest)).andReturn(changeRequest); + expect(dns.applyChangeRequest(ZONE_NAME, changeRequest, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.START_TIME))) + .andReturn(changeRequest); + replay(dns); + assertSame(changeRequest, changeRequest.applyTo()); + assertSame(changeRequest, + changeRequest.applyTo(Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.START_TIME))); + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/DnsImplTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/DnsImplTest.java new file mode 100644 index 000000000000..94ed4a3da3f7 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/DnsImplTest.java @@ -0,0 +1,388 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import com.google.api.services.dns.model.Change; +import com.google.api.services.dns.model.ManagedZone; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Lists; +import com.google.gcloud.Page; +import com.google.gcloud.RetryParams; +import com.google.gcloud.ServiceOptions; +import com.google.gcloud.dns.spi.DnsRpc; +import com.google.gcloud.dns.spi.DnsRpcFactory; + +import org.easymock.Capture; +import org.easymock.EasyMock; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import java.util.Map; + +public class DnsImplTest { + + // Dns entities + private static final String ZONE_NAME = "some zone name"; + private static final String DNS_NAME = "example.com."; + private static final String DESCRIPTION = "desc"; + private static final String CHANGE_ID = "some change id"; + private static final RecordSet DNS_RECORD1 = + RecordSet.builder("Something", RecordSet.Type.AAAA).build(); + private static final RecordSet DNS_RECORD2 = + RecordSet.builder("Different", RecordSet.Type.AAAA).build(); + private static final Integer MAX_SIZE = 20; + private static final String PAGE_TOKEN = "some token"; + private static final ZoneInfo ZONE_INFO = ZoneInfo.of(ZONE_NAME, DNS_NAME, DESCRIPTION); + private static final ProjectInfo PROJECT_INFO = ProjectInfo.builder().build(); + private static final ChangeRequestInfo CHANGE_REQUEST_PARTIAL = ChangeRequestInfo.builder() + .add(DNS_RECORD1) + .build(); + private static final ChangeRequestInfo CHANGE_REQUEST_COMPLETE = ChangeRequestInfo.builder() + .add(DNS_RECORD1) + .startTimeMillis(123L) + .status(ChangeRequest.Status.PENDING) + .id(CHANGE_ID) + .build(); + + // Result lists + private static final DnsRpc.ListResult LIST_RESULT_OF_PB_CHANGES = + DnsRpc.ListResult.of("cursor", ImmutableList.of(CHANGE_REQUEST_COMPLETE.toPb(), + CHANGE_REQUEST_PARTIAL.toPb())); + private static final DnsRpc.ListResult LIST_RESULT_OF_PB_ZONES = + DnsRpc.ListResult.of("cursor", ImmutableList.of(ZONE_INFO.toPb())); + private static final DnsRpc.ListResult + LIST_OF_PB_DNS_RECORDS = + DnsRpc.ListResult.of("cursor", ImmutableList.of(DNS_RECORD1.toPb(), DNS_RECORD2.toPb())); + + // Field options + private static final Dns.ZoneOption ZONE_FIELDS = + Dns.ZoneOption.fields(Dns.ZoneField.CREATION_TIME); + private static final Dns.ProjectOption PROJECT_FIELDS = + Dns.ProjectOption.fields(Dns.ProjectField.QUOTA); + private static final Dns.ChangeRequestOption CHANGE_GET_FIELDS = + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS); + + // Listing options + private static final Dns.ZoneListOption[] ZONE_LIST_OPTIONS = { + Dns.ZoneListOption.pageSize(MAX_SIZE), Dns.ZoneListOption.pageToken(PAGE_TOKEN), + Dns.ZoneListOption.fields(Dns.ZoneField.DESCRIPTION), + Dns.ZoneListOption.dnsName(DNS_NAME)}; + private static final Dns.ChangeRequestListOption[] CHANGE_LIST_OPTIONS = { + Dns.ChangeRequestListOption.pageSize(MAX_SIZE), + Dns.ChangeRequestListOption.pageToken(PAGE_TOKEN), + Dns.ChangeRequestListOption.fields(Dns.ChangeRequestField.STATUS), + Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.ASCENDING)}; + private static final Dns.RecordSetListOption[] DNS_RECORD_LIST_OPTIONS = { + Dns.RecordSetListOption.pageSize(MAX_SIZE), + Dns.RecordSetListOption.pageToken(PAGE_TOKEN), + Dns.RecordSetListOption.fields(Dns.RecordSetField.TTL), + Dns.RecordSetListOption.dnsName(DNS_NAME), + Dns.RecordSetListOption.type(RecordSet.Type.AAAA)}; + + // Other + private static final Map EMPTY_RPC_OPTIONS = ImmutableMap.of(); + private static final ServiceOptions.Clock TIME_SOURCE = new ServiceOptions.Clock() { + @Override + public long millis() { + return 42000L; + } + }; + + private DnsOptions options; + private DnsRpcFactory rpcFactoryMock; + private DnsRpc dnsRpcMock; + private Dns dns; + + @Before + public void setUp() { + rpcFactoryMock = EasyMock.createMock(DnsRpcFactory.class); + dnsRpcMock = EasyMock.createMock(DnsRpc.class); + EasyMock.expect(rpcFactoryMock.create(EasyMock.anyObject(DnsOptions.class))) + .andReturn(dnsRpcMock); + EasyMock.replay(rpcFactoryMock); + options = DnsOptions.builder() + .projectId("projectId") + .clock(TIME_SOURCE) + .serviceRpcFactory(rpcFactoryMock) + .retryParams(RetryParams.noRetries()) + .build(); + } + + @After + public void tearDown() throws Exception { + EasyMock.verify(rpcFactoryMock); + } + + @Test + public void testCreateZone() { + EasyMock.expect(dnsRpcMock.create(ZONE_INFO.toPb(), EMPTY_RPC_OPTIONS)) + .andReturn(ZONE_INFO.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Zone zone = dns.create(ZONE_INFO); + assertEquals(new Zone(dns, new ZoneInfo.BuilderImpl(ZONE_INFO)), zone); + } + + @Test + public void testCreateZoneWithOptions() { + Capture> capturedOptions = Capture.newInstance(); + EasyMock.expect(dnsRpcMock.create(EasyMock.eq(ZONE_INFO.toPb()), + EasyMock.capture(capturedOptions))).andReturn(ZONE_INFO.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Zone zone = dns.create(ZONE_INFO, ZONE_FIELDS); + String selector = (String) capturedOptions.getValue().get(ZONE_FIELDS.rpcOption()); + assertEquals(new Zone(dns, new ZoneInfo.BuilderImpl(ZONE_INFO)), zone); + assertTrue(selector.contains(Dns.ZoneField.CREATION_TIME.selector())); + assertTrue(selector.contains(Dns.ZoneField.NAME.selector())); + } + + @Test + public void testGetZone() { + EasyMock.expect(dnsRpcMock.getZone(ZONE_INFO.name(), EMPTY_RPC_OPTIONS)) + .andReturn(ZONE_INFO.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Zone zone = dns.getZone(ZONE_INFO.name()); + assertEquals(new Zone(dns, new ZoneInfo.BuilderImpl(ZONE_INFO)), zone); + } + + @Test + public void testGetZoneWithOptions() { + Capture> capturedOptions = Capture.newInstance(); + EasyMock.expect(dnsRpcMock.getZone(EasyMock.eq(ZONE_INFO.name()), + EasyMock.capture(capturedOptions))).andReturn(ZONE_INFO.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Zone zone = dns.getZone(ZONE_INFO.name(), ZONE_FIELDS); + String selector = (String) capturedOptions.getValue().get(ZONE_FIELDS.rpcOption()); + assertEquals(new Zone(dns, new ZoneInfo.BuilderImpl(ZONE_INFO)), zone); + assertTrue(selector.contains(Dns.ZoneField.CREATION_TIME.selector())); + assertTrue(selector.contains(Dns.ZoneField.NAME.selector())); + } + + @Test + public void testDeleteZone() { + EasyMock.expect(dnsRpcMock.deleteZone(ZONE_INFO.name())) + .andReturn(true); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + assertTrue(dns.delete(ZONE_INFO.name())); + } + + @Test + public void testGetProject() { + EasyMock.expect(dnsRpcMock.getProject(EMPTY_RPC_OPTIONS)) + .andReturn(PROJECT_INFO.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + ProjectInfo projectInfo = dns.getProject(); + assertEquals(PROJECT_INFO, projectInfo); + } + + @Test + public void testProjectGetWithOptions() { + Capture> capturedOptions = Capture.newInstance(); + EasyMock.expect(dnsRpcMock.getProject(EasyMock.capture(capturedOptions))) + .andReturn(PROJECT_INFO.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + ProjectInfo projectInfo = dns.getProject(PROJECT_FIELDS); + String selector = (String) capturedOptions.getValue().get(PROJECT_FIELDS.rpcOption()); + assertEquals(PROJECT_INFO, projectInfo); + assertTrue(selector.contains(Dns.ProjectField.QUOTA.selector())); + assertTrue(selector.contains(Dns.ProjectField.PROJECT_ID.selector())); + } + + @Test + public void testGetChangeRequest() { + EasyMock.expect(dnsRpcMock.getChangeRequest(ZONE_INFO.name(), CHANGE_REQUEST_COMPLETE.id(), + EMPTY_RPC_OPTIONS)).andReturn(CHANGE_REQUEST_COMPLETE.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + ChangeRequest changeRequest = dns.getChangeRequest(ZONE_INFO.name(), + CHANGE_REQUEST_COMPLETE.id()); + assertEquals(new ChangeRequest(dns, ZONE_INFO.name(), + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_COMPLETE)), changeRequest); + } + + @Test + public void testGetChangeRequestWithOptions() { + Capture> capturedOptions = Capture.newInstance(); + EasyMock.expect(dnsRpcMock.getChangeRequest(EasyMock.eq(ZONE_INFO.name()), + EasyMock.eq(CHANGE_REQUEST_COMPLETE.id()), EasyMock.capture(capturedOptions))) + .andReturn(CHANGE_REQUEST_COMPLETE.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + ChangeRequest changeRequest = dns.getChangeRequest(ZONE_INFO.name(), + CHANGE_REQUEST_COMPLETE.id(), CHANGE_GET_FIELDS); + String selector = (String) capturedOptions.getValue().get(CHANGE_GET_FIELDS.rpcOption()); + assertEquals(new ChangeRequest(dns, ZONE_INFO.name(), + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_COMPLETE)), changeRequest); + assertTrue(selector.contains(Dns.ChangeRequestField.STATUS.selector())); + assertTrue(selector.contains(Dns.ChangeRequestField.ID.selector())); + } + + @Test + public void testApplyChangeRequest() { + EasyMock.expect(dnsRpcMock.applyChangeRequest(ZONE_INFO.name(), CHANGE_REQUEST_PARTIAL.toPb(), + EMPTY_RPC_OPTIONS)).andReturn(CHANGE_REQUEST_COMPLETE.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + ChangeRequest changeRequest = dns.applyChangeRequest(ZONE_INFO.name(), + CHANGE_REQUEST_PARTIAL); + assertEquals(new ChangeRequest(dns, ZONE_INFO.name(), + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_COMPLETE)), changeRequest); + } + + @Test + public void testApplyChangeRequestWithOptions() { + Capture> capturedOptions = Capture.newInstance(); + EasyMock.expect(dnsRpcMock.applyChangeRequest(EasyMock.eq(ZONE_INFO.name()), + EasyMock.eq(CHANGE_REQUEST_PARTIAL.toPb()), EasyMock.capture(capturedOptions))) + .andReturn(CHANGE_REQUEST_COMPLETE.toPb()); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + ChangeRequest changeRequest = dns.applyChangeRequest(ZONE_INFO.name(), + CHANGE_REQUEST_PARTIAL, CHANGE_GET_FIELDS); + String selector = (String) capturedOptions.getValue().get(CHANGE_GET_FIELDS.rpcOption()); + assertEquals(new ChangeRequest(dns, ZONE_INFO.name(), + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_COMPLETE)), changeRequest); + assertTrue(selector.contains(Dns.ChangeRequestField.STATUS.selector())); + assertTrue(selector.contains(Dns.ChangeRequestField.ID.selector())); + } + + // lists + @Test + public void testListChangeRequests() { + EasyMock.expect(dnsRpcMock.listChangeRequests(ZONE_INFO.name(), EMPTY_RPC_OPTIONS)) + .andReturn(LIST_RESULT_OF_PB_CHANGES); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Page changeRequestPage = dns.listChangeRequests(ZONE_INFO.name()); + assertTrue(Lists.newArrayList(changeRequestPage.values()).contains( + new ChangeRequest(dns, ZONE_INFO.name(), + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_COMPLETE)))); + assertTrue(Lists.newArrayList(changeRequestPage.values()).contains( + new ChangeRequest(dns, ZONE_INFO.name(), + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_PARTIAL)))); + assertEquals(2, Lists.newArrayList(changeRequestPage.values()).size()); + } + + @Test + public void testListChangeRequestsWithOptions() { + Capture> capturedOptions = Capture.newInstance(); + EasyMock.expect(dnsRpcMock.listChangeRequests(EasyMock.eq(ZONE_NAME), + EasyMock.capture(capturedOptions))).andReturn(LIST_RESULT_OF_PB_CHANGES); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Page changeRequestPage = dns.listChangeRequests(ZONE_NAME, CHANGE_LIST_OPTIONS); + assertTrue(Lists.newArrayList(changeRequestPage.values()).contains( + new ChangeRequest(dns, ZONE_INFO.name(), + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_COMPLETE)))); + assertTrue(Lists.newArrayList(changeRequestPage.values()).contains( + new ChangeRequest(dns, ZONE_INFO.name(), + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_PARTIAL)))); + assertEquals(2, Lists.newArrayList(changeRequestPage.values()).size()); + Integer size = (Integer) capturedOptions.getValue().get(CHANGE_LIST_OPTIONS[0].rpcOption()); + assertEquals(MAX_SIZE, size); + String selector = (String) capturedOptions.getValue().get(CHANGE_LIST_OPTIONS[1].rpcOption()); + assertEquals(PAGE_TOKEN, selector); + selector = (String) capturedOptions.getValue().get(CHANGE_LIST_OPTIONS[2].rpcOption()); + assertTrue(selector.contains(Dns.ChangeRequestField.STATUS.selector())); + assertTrue(selector.contains(Dns.ChangeRequestField.ID.selector())); + selector = (String) capturedOptions.getValue().get(CHANGE_LIST_OPTIONS[3].rpcOption()); + assertTrue(selector.contains(Dns.SortingOrder.ASCENDING.selector())); + } + + @Test + public void testListZones() { + EasyMock.expect(dnsRpcMock.listZones(EMPTY_RPC_OPTIONS)) + .andReturn(LIST_RESULT_OF_PB_ZONES); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Page zonePage = dns.listZones(); + assertEquals(1, Lists.newArrayList(zonePage.values()).size()); + assertEquals(new Zone(dns, new ZoneInfo.BuilderImpl(ZONE_INFO)), + Lists.newArrayList(zonePage.values()).get(0)); + } + + @Test + public void testListZonesWithOptions() { + Capture> capturedOptions = Capture.newInstance(); + EasyMock.expect(dnsRpcMock.listZones(EasyMock.capture(capturedOptions))) + .andReturn(LIST_RESULT_OF_PB_ZONES); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Page zonePage = dns.listZones(ZONE_LIST_OPTIONS); + assertEquals(1, Lists.newArrayList(zonePage.values()).size()); + assertEquals(new Zone(dns, new ZoneInfo.BuilderImpl(ZONE_INFO)), + Lists.newArrayList(zonePage.values()).get(0)); + Integer size = (Integer) capturedOptions.getValue().get(ZONE_LIST_OPTIONS[0].rpcOption()); + assertEquals(MAX_SIZE, size); + String selector = (String) capturedOptions.getValue().get(ZONE_LIST_OPTIONS[1].rpcOption()); + assertEquals(PAGE_TOKEN, selector); + selector = (String) capturedOptions.getValue().get(ZONE_LIST_OPTIONS[2].rpcOption()); + assertTrue(selector.contains(Dns.ZoneField.DESCRIPTION.selector())); + assertTrue(selector.contains(Dns.ZoneField.NAME.selector())); + selector = (String) capturedOptions.getValue().get(ZONE_LIST_OPTIONS[3].rpcOption()); + assertEquals(DNS_NAME, selector); + } + + @Test + public void testListDnsRecords() { + EasyMock.expect(dnsRpcMock.listRecordSets(ZONE_INFO.name(), EMPTY_RPC_OPTIONS)) + .andReturn(LIST_OF_PB_DNS_RECORDS); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Page dnsPage = dns.listRecordSets(ZONE_INFO.name()); + assertEquals(2, Lists.newArrayList(dnsPage.values()).size()); + assertTrue(Lists.newArrayList(dnsPage.values()).contains(DNS_RECORD1)); + assertTrue(Lists.newArrayList(dnsPage.values()).contains(DNS_RECORD2)); + } + + @Test + public void testListDnsRecordsWithOptions() { + Capture> capturedOptions = Capture.newInstance(); + EasyMock.expect(dnsRpcMock.listRecordSets(EasyMock.eq(ZONE_NAME), + EasyMock.capture(capturedOptions))).andReturn(LIST_OF_PB_DNS_RECORDS); + EasyMock.replay(dnsRpcMock); + dns = options.service(); // creates DnsImpl + Page dnsPage = dns.listRecordSets(ZONE_NAME, DNS_RECORD_LIST_OPTIONS); + assertEquals(2, Lists.newArrayList(dnsPage.values()).size()); + assertTrue(Lists.newArrayList(dnsPage.values()).contains(DNS_RECORD1)); + assertTrue(Lists.newArrayList(dnsPage.values()).contains(DNS_RECORD2)); + Integer size = (Integer) capturedOptions.getValue().get(DNS_RECORD_LIST_OPTIONS[0].rpcOption()); + assertEquals(MAX_SIZE, size); + String selector = (String) capturedOptions.getValue() + .get(DNS_RECORD_LIST_OPTIONS[1].rpcOption()); + assertEquals(PAGE_TOKEN, selector); + selector = (String) capturedOptions.getValue().get(DNS_RECORD_LIST_OPTIONS[2].rpcOption()); + assertTrue(selector.contains(Dns.RecordSetField.NAME.selector())); + assertTrue(selector.contains(Dns.RecordSetField.TTL.selector())); + selector = (String) capturedOptions.getValue().get(DNS_RECORD_LIST_OPTIONS[3].rpcOption()); + assertEquals(DNS_RECORD_LIST_OPTIONS[3].value(), selector); + String type = (String) capturedOptions.getValue().get(DNS_RECORD_LIST_OPTIONS[4] + .rpcOption()); + assertEquals(DNS_RECORD_LIST_OPTIONS[4].value(), type); + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/DnsTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/DnsTest.java new file mode 100644 index 000000000000..df86d6ebd495 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/DnsTest.java @@ -0,0 +1,146 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; + +import com.google.gcloud.dns.spi.DnsRpc; + +import org.junit.Test; + +public class DnsTest { + + private static final Integer PAGE_SIZE = 20; + private static final String PAGE_TOKEN = "page token"; + private static final String DNS_NAME = "www.example.com."; + + @Test + public void testRecordSetListOption() { + // dns name + String dnsName = "some name"; + Dns.RecordSetListOption recordSetListOption = Dns.RecordSetListOption.dnsName(dnsName); + assertEquals(dnsName, recordSetListOption.value()); + assertEquals(DnsRpc.Option.NAME, recordSetListOption.rpcOption()); + // page token + recordSetListOption = Dns.RecordSetListOption.pageToken(PAGE_TOKEN); + assertEquals(PAGE_TOKEN, recordSetListOption.value()); + assertEquals(DnsRpc.Option.PAGE_TOKEN, recordSetListOption.rpcOption()); + // page size + recordSetListOption = Dns.RecordSetListOption.pageSize(PAGE_SIZE); + assertEquals(PAGE_SIZE, recordSetListOption.value()); + assertEquals(DnsRpc.Option.PAGE_SIZE, recordSetListOption.rpcOption()); + // record type + RecordSet.Type recordType = RecordSet.Type.AAAA; + recordSetListOption = Dns.RecordSetListOption.type(recordType); + assertEquals(recordType.name(), recordSetListOption.value()); + assertEquals(DnsRpc.Option.DNS_TYPE, recordSetListOption.rpcOption()); + // fields + recordSetListOption = Dns.RecordSetListOption.fields(Dns.RecordSetField.NAME, + Dns.RecordSetField.TTL); + assertEquals(DnsRpc.Option.FIELDS, recordSetListOption.rpcOption()); + assertTrue(recordSetListOption.value() instanceof String); + assertTrue(((String) recordSetListOption.value()).contains( + Dns.RecordSetField.NAME.selector())); + assertTrue(((String) recordSetListOption.value()).contains( + Dns.RecordSetField.TTL.selector())); + assertTrue(((String) recordSetListOption.value()).contains( + Dns.RecordSetField.NAME.selector())); + } + + @Test + public void testZoneOption() { + Dns.ZoneOption fields = Dns.ZoneOption.fields(Dns.ZoneField.CREATION_TIME, + Dns.ZoneField.DESCRIPTION); + assertEquals(DnsRpc.Option.FIELDS, fields.rpcOption()); + assertTrue(fields.value() instanceof String); + assertTrue(((String) fields.value()).contains(Dns.ZoneField.CREATION_TIME.selector())); + assertTrue(((String) fields.value()).contains(Dns.ZoneField.DESCRIPTION.selector())); + } + + @Test + public void testZoneList() { + // fields + Dns.ZoneListOption fields = Dns.ZoneListOption.fields(Dns.ZoneField.CREATION_TIME, + Dns.ZoneField.DESCRIPTION); + assertEquals(DnsRpc.Option.FIELDS, fields.rpcOption()); + assertTrue(fields.value() instanceof String); + assertTrue(((String) fields.value()).contains(Dns.ZoneField.CREATION_TIME.selector())); + assertTrue(((String) fields.value()).contains(Dns.ZoneField.DESCRIPTION.selector())); + assertTrue(((String) fields.value()).contains(Dns.ZoneField.NAME.selector())); + // page token + Dns.ZoneListOption option = Dns.ZoneListOption.pageToken(PAGE_TOKEN); + assertEquals(PAGE_TOKEN, option.value()); + assertEquals(DnsRpc.Option.PAGE_TOKEN, option.rpcOption()); + // page size + option = Dns.ZoneListOption.pageSize(PAGE_SIZE); + assertEquals(PAGE_SIZE, option.value()); + assertEquals(DnsRpc.Option.PAGE_SIZE, option.rpcOption()); + // dnsName filter + option = Dns.ZoneListOption.dnsName(DNS_NAME); + assertEquals(DNS_NAME, option.value()); + assertEquals(DnsRpc.Option.DNS_NAME, option.rpcOption()); + } + + @Test + public void testProjectGetOption() { + // fields + Dns.ProjectOption fields = Dns.ProjectOption.fields(Dns.ProjectField.QUOTA); + assertEquals(DnsRpc.Option.FIELDS, fields.rpcOption()); + assertTrue(fields.value() instanceof String); + assertTrue(((String) fields.value()).contains(Dns.ProjectField.QUOTA.selector())); + assertTrue(((String) fields.value()).contains(Dns.ProjectField.PROJECT_ID.selector())); + } + + @Test + public void testChangeRequestOption() { + // fields + Dns.ChangeRequestOption fields = Dns.ChangeRequestOption.fields( + Dns.ChangeRequestField.START_TIME, Dns.ChangeRequestField.STATUS); + assertEquals(DnsRpc.Option.FIELDS, fields.rpcOption()); + assertTrue(fields.value() instanceof String); + assertTrue(((String) fields.value()).contains( + Dns.ChangeRequestField.START_TIME.selector())); + assertTrue(((String) fields.value()).contains(Dns.ChangeRequestField.STATUS.selector())); + assertTrue(((String) fields.value()).contains(Dns.ChangeRequestField.ID.selector())); + } + + @Test + public void testChangeRequestListOption() { + // fields + Dns.ChangeRequestListOption fields = Dns.ChangeRequestListOption.fields( + Dns.ChangeRequestField.START_TIME, Dns.ChangeRequestField.STATUS); + assertEquals(DnsRpc.Option.FIELDS, fields.rpcOption()); + assertTrue(fields.value() instanceof String); + assertTrue(((String) fields.value()).contains( + Dns.ChangeRequestField.START_TIME.selector())); + assertTrue(((String) fields.value()).contains(Dns.ChangeRequestField.STATUS.selector())); + assertTrue(((String) fields.value()).contains(Dns.ChangeRequestField.ID.selector())); + // page token + Dns.ChangeRequestListOption option = Dns.ChangeRequestListOption.pageToken(PAGE_TOKEN); + assertEquals(PAGE_TOKEN, option.value()); + assertEquals(DnsRpc.Option.PAGE_TOKEN, option.rpcOption()); + // page size + option = Dns.ChangeRequestListOption.pageSize(PAGE_SIZE); + assertEquals(PAGE_SIZE, option.value()); + assertEquals(DnsRpc.Option.PAGE_SIZE, option.rpcOption()); + // sort order + option = Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.ASCENDING); + assertEquals(DnsRpc.Option.SORTING_ORDER, option.rpcOption()); + assertEquals(Dns.SortingOrder.ASCENDING.selector(), option.value()); + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ProjectInfoTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ProjectInfoTest.java new file mode 100644 index 000000000000..d959d44d4351 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ProjectInfoTest.java @@ -0,0 +1,118 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotEquals; +import static org.junit.Assert.assertNull; + +import org.junit.Test; + +import java.math.BigInteger; + +public class ProjectInfoTest { + + private static final String ID = "project-id-123"; + private static final BigInteger NUMBER = new BigInteger("123"); + private static final ProjectInfo.Quota QUOTA = new ProjectInfo.Quota(1, 2, 3, 4, 5, 6); + private static final ProjectInfo PROJECT_INFO = ProjectInfo.builder() + .id(ID).number(NUMBER).quota(QUOTA).build(); + + @Test + public void testBuilder() { + ProjectInfo withId = ProjectInfo.builder().id(ID).build(); + assertEquals(ID, withId.id()); + assertNull(withId.number()); + assertNull(withId.quota()); + ProjectInfo withNumber = ProjectInfo.builder().number(NUMBER).build(); + assertEquals(NUMBER, withNumber.number()); + assertNull(withNumber.quota()); + assertNull(withNumber.id()); + ProjectInfo withQuota = ProjectInfo.builder().quota(QUOTA).build(); + assertEquals(QUOTA, withQuota.quota()); + assertNull(withQuota.id()); + assertNull(withQuota.number()); + assertEquals(QUOTA, PROJECT_INFO.quota()); + assertEquals(NUMBER, PROJECT_INFO.number()); + assertEquals(ID, PROJECT_INFO.id()); + } + + @Test + public void testQuotaConstructor() { + assertEquals(1, QUOTA.zones()); + assertEquals(2, QUOTA.resourceRecordsPerRrset()); + assertEquals(3, QUOTA.rrsetAdditionsPerChange()); + assertEquals(4, QUOTA.rrsetDeletionsPerChange()); + assertEquals(5, QUOTA.rrsetsPerZone()); + assertEquals(6, QUOTA.totalRrdataSizePerChange()); + } + + @Test + public void testEqualsAndNotEqualsQuota() { + ProjectInfo.Quota clone = new ProjectInfo.Quota(6, 5, 4, 3, 2, 1); + assertNotEquals(QUOTA, clone); + clone = ProjectInfo.Quota.fromPb(QUOTA.toPb()); + assertEquals(QUOTA, clone); + } + + @Test + public void testSameHashCodeOnEqualsQuota() { + ProjectInfo.Quota clone = ProjectInfo.Quota.fromPb(QUOTA.toPb()); + assertEquals(QUOTA, clone); + assertEquals(QUOTA.hashCode(), clone.hashCode()); + } + + @Test + public void testEqualsAndNotEquals() { + ProjectInfo clone = ProjectInfo.builder().build(); + assertNotEquals(PROJECT_INFO, clone); + clone = ProjectInfo.builder().id(PROJECT_INFO.id()).number(PROJECT_INFO.number()).build(); + assertNotEquals(PROJECT_INFO, clone); + clone = ProjectInfo.builder().id(PROJECT_INFO.id()).quota(PROJECT_INFO.quota()).build(); + assertNotEquals(PROJECT_INFO, clone); + clone = ProjectInfo.builder().number(PROJECT_INFO.number()).quota(PROJECT_INFO.quota()).build(); + assertNotEquals(PROJECT_INFO, clone); + clone = ProjectInfo.fromPb(PROJECT_INFO.toPb()); + assertEquals(PROJECT_INFO, clone); + } + + @Test + public void testSameHashCodeOnEquals() { + ProjectInfo clone = ProjectInfo.fromPb(PROJECT_INFO.toPb()); + assertEquals(PROJECT_INFO, clone); + assertEquals(PROJECT_INFO.hashCode(), clone.hashCode()); + } + + @Test + public void testToAndFromPb() { + assertEquals(PROJECT_INFO, ProjectInfo.fromPb(PROJECT_INFO.toPb())); + ProjectInfo partial = ProjectInfo.builder().id(ID).build(); + assertEquals(partial, ProjectInfo.fromPb(partial.toPb())); + partial = ProjectInfo.builder().number(NUMBER).build(); + assertEquals(partial, ProjectInfo.fromPb(partial.toPb())); + partial = ProjectInfo.builder().quota(QUOTA).build(); + assertEquals(partial, ProjectInfo.fromPb(partial.toPb())); + assertNotEquals(PROJECT_INFO, partial); + } + + @Test + public void testToAndFromPbQuota() { + assertEquals(QUOTA, ProjectInfo.Quota.fromPb(QUOTA.toPb())); + ProjectInfo.Quota wrong = new ProjectInfo.Quota(5, 6, 3, 6, 2, 1); + assertNotEquals(QUOTA, ProjectInfo.Quota.fromPb(wrong.toPb())); + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/RecordSetTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/RecordSetTest.java new file mode 100644 index 000000000000..369e078a48c7 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/RecordSetTest.java @@ -0,0 +1,150 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static com.google.gcloud.dns.RecordSet.builder; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import org.junit.Test; + +import java.util.concurrent.TimeUnit; + +public class RecordSetTest { + + private static final String NAME = "example.com."; + private static final Integer TTL = 3600; + private static final TimeUnit UNIT = TimeUnit.HOURS; + private static final Integer UNIT_TTL = 1; + private static final RecordSet.Type TYPE = RecordSet.Type.AAAA; + private static final RecordSet recordSet = builder(NAME, TYPE) + .ttl(UNIT_TTL, UNIT) + .build(); + + @Test + public void testDefaultDnsRecord() { + RecordSet recordSet = builder(NAME, TYPE).build(); + assertEquals(0, recordSet.records().size()); + assertEquals(TYPE, recordSet.type()); + assertEquals(NAME, recordSet.name()); + } + + @Test + public void testBuilder() { + assertEquals(NAME, recordSet.name()); + assertEquals(TTL, recordSet.ttl()); + assertEquals(TYPE, recordSet.type()); + assertEquals(0, recordSet.records().size()); + // verify that one can add records to the record set + String testingRecord = "Testing recordSet"; + String anotherTestingRecord = "Another recordSet 123"; + RecordSet anotherRecord = recordSet.toBuilder() + .addRecord(testingRecord) + .addRecord(anotherTestingRecord) + .build(); + assertEquals(2, anotherRecord.records().size()); + assertTrue(anotherRecord.records().contains(testingRecord)); + assertTrue(anotherRecord.records().contains(anotherTestingRecord)); + } + + @Test + public void testValidTtl() { + try { + builder(NAME, TYPE).ttl(-1, TimeUnit.SECONDS); + fail("A negative value is not acceptable for ttl."); + } catch (IllegalArgumentException e) { + // expected + } + builder(NAME, TYPE).ttl(0, TimeUnit.SECONDS); + builder(NAME, TYPE).ttl(Integer.MAX_VALUE, TimeUnit.SECONDS); + try { + builder(NAME, TYPE).ttl(Integer.MAX_VALUE, TimeUnit.HOURS); + fail("This value is too large for int."); + } catch (IllegalArgumentException e) { + // expected + } + RecordSet record = RecordSet.builder(NAME, TYPE).ttl(UNIT_TTL, UNIT).build(); + assertEquals(TTL, record.ttl()); + } + + @Test + public void testEqualsAndNotEquals() { + RecordSet clone = recordSet.toBuilder().build(); + assertEquals(recordSet, clone); + clone = recordSet.toBuilder().addRecord("another recordSet").build(); + assertNotEquals(recordSet, clone); + String differentName = "totally different name"; + clone = recordSet.toBuilder().name(differentName).build(); + assertNotEquals(recordSet, clone); + clone = recordSet.toBuilder().ttl(recordSet.ttl() + 1, TimeUnit.SECONDS).build(); + assertNotEquals(recordSet, clone); + clone = recordSet.toBuilder().type(RecordSet.Type.TXT).build(); + assertNotEquals(recordSet, clone); + } + + @Test + public void testSameHashCodeOnEquals() { + int hash = recordSet.hashCode(); + RecordSet clone = recordSet.toBuilder().build(); + assertEquals(clone.hashCode(), hash); + } + + @Test + public void testToAndFromPb() { + assertEquals(recordSet, RecordSet.fromPb(recordSet.toPb())); + RecordSet partial = builder(NAME, TYPE).build(); + assertEquals(partial, RecordSet.fromPb(partial.toPb())); + partial = builder(NAME, TYPE).addRecord("test").build(); + assertEquals(partial, RecordSet.fromPb(partial.toPb())); + partial = builder(NAME, TYPE).ttl(15, TimeUnit.SECONDS).build(); + assertEquals(partial, RecordSet.fromPb(partial.toPb())); + } + + @Test + public void testToBuilder() { + assertEquals(recordSet, recordSet.toBuilder().build()); + RecordSet partial = builder(NAME, TYPE).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = builder(NAME, TYPE).addRecord("test").build(); + assertEquals(partial, partial.toBuilder().build()); + partial = builder(NAME, TYPE).ttl(15, TimeUnit.SECONDS).build(); + assertEquals(partial, partial.toBuilder().build()); + } + + @Test + public void clearRecordSet() { + // make sure that we are starting not empty + RecordSet clone = + recordSet.toBuilder().addRecord("record").addRecord("another").build(); + assertNotEquals(0, clone.records().size()); + clone = clone.toBuilder().clearRecords().build(); + assertEquals(0, clone.records().size()); + clone.toPb(); // verify that pb allows it + } + + @Test + public void removeFromRecordSet() { + String recordString = "record"; + // make sure that we are starting not empty + RecordSet clone = recordSet.toBuilder().addRecord(recordString).build(); + assertNotEquals(0, clone.records().size()); + clone = clone.toBuilder().removeRecord(recordString).build(); + assertEquals(0, clone.records().size()); + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/SerializationTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/SerializationTest.java new file mode 100644 index 000000000000..ad25b31068dd --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/SerializationTest.java @@ -0,0 +1,108 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import com.google.common.collect.ImmutableList; +import com.google.gcloud.AuthCredentials; +import com.google.gcloud.BaseSerializationTest; +import com.google.gcloud.Restorable; +import com.google.gcloud.RetryParams; + +import java.io.Serializable; +import java.math.BigInteger; +import java.util.concurrent.TimeUnit; + +public class SerializationTest extends BaseSerializationTest { + + private static final ZoneInfo FULL_ZONE_INFO = Zone.of("some zone name", "www.example.com", + "some descriptions").toBuilder() + .creationTimeMillis(132L) + .id("123333") + .nameServers(ImmutableList.of("server 1", "server 2")) + .nameServerSet("specificationstring") + .build(); + private static final ZoneInfo PARTIAL_ZONE_INFO = Zone.of("some zone name", "www.example.com", + "some descriptions").toBuilder().build(); + private static final ProjectInfo PARTIAL_PROJECT_INFO = ProjectInfo.builder().id("13").build(); + private static final ProjectInfo FULL_PROJECT_INFO = ProjectInfo.builder() + .id("342") + .number(new BigInteger("2343245")) + .quota(new ProjectInfo.Quota(12, 13, 14, 15, 16, 17)) + .build(); + private static final Dns.ZoneListOption ZONE_LIST_OPTION = + Dns.ZoneListOption.dnsName("www.example.com."); + private static final Dns.RecordSetListOption RECORD_SET_LIST_OPTION = + Dns.RecordSetListOption.fields(Dns.RecordSetField.TTL); + private static final Dns.ChangeRequestListOption CHANGE_REQUEST_LIST_OPTION = + Dns.ChangeRequestListOption.fields(Dns.ChangeRequestField.STATUS); + private static final Dns.ZoneOption ZONE_OPTION = + Dns.ZoneOption.fields(Dns.ZoneField.CREATION_TIME); + private static final Dns.ChangeRequestOption CHANGE_REQUEST_OPTION = + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS); + private static final Dns.ProjectOption PROJECT_OPTION = + Dns.ProjectOption.fields(Dns.ProjectField.QUOTA); + private static final DnsOptions OPTIONS = DnsOptions.builder() + .projectId("some-unnecessary-project-ID") + .retryParams(RetryParams.defaultInstance()) + .build(); + private static final Dns DNS = OPTIONS.service(); + private static final Zone FULL_ZONE = new Zone(DNS, new ZoneInfo.BuilderImpl(FULL_ZONE_INFO)); + private static final Zone PARTIAL_ZONE = + new Zone(DNS, new ZoneInfo.BuilderImpl(PARTIAL_ZONE_INFO)); + private static final ChangeRequestInfo CHANGE_REQUEST_INFO_PARTIAL = + ChangeRequest.builder().build(); + private static final ChangeRequest CHANGE_REQUEST_PARTIAL = new ChangeRequest(DNS, "name", + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_INFO_PARTIAL)); + private static final RecordSet RECORD_SET_PARTIAL = + RecordSet.builder("www.www.com", RecordSet.Type.AAAA).build(); + private static final RecordSet RECORD_SET_COMPLETE = + RecordSet.builder("www.sadfa.com", RecordSet.Type.A) + .ttl(12, TimeUnit.HOURS) + .addRecord("record") + .build(); + private static final ChangeRequestInfo CHANGE_REQUEST_INFO_COMPLETE = ChangeRequestInfo.builder() + .add(RECORD_SET_COMPLETE) + .delete(RECORD_SET_PARTIAL) + .status(ChangeRequest.Status.PENDING) + .id("some id") + .startTimeMillis(132L) + .build(); + private static final ChangeRequest CHANGE_REQUEST_COMPLETE = new ChangeRequest(DNS, "name", + new ChangeRequestInfo.BuilderImpl(CHANGE_REQUEST_INFO_COMPLETE)); + + @Override + protected Serializable[] serializableObjects() { + DnsOptions options = DnsOptions.builder() + .authCredentials(AuthCredentials.createForAppEngine()) + .projectId("id1") + .build(); + DnsOptions otherOptions = options.toBuilder() + .authCredentials(null) + .build(); + return new Serializable[]{FULL_ZONE_INFO, PARTIAL_ZONE_INFO, ZONE_LIST_OPTION, + RECORD_SET_LIST_OPTION, CHANGE_REQUEST_LIST_OPTION, ZONE_OPTION, CHANGE_REQUEST_OPTION, + PROJECT_OPTION, PARTIAL_PROJECT_INFO, FULL_PROJECT_INFO, OPTIONS, FULL_ZONE, PARTIAL_ZONE, + OPTIONS, CHANGE_REQUEST_INFO_PARTIAL, CHANGE_REQUEST_PARTIAL, RECORD_SET_PARTIAL, + RECORD_SET_COMPLETE, CHANGE_REQUEST_INFO_COMPLETE, CHANGE_REQUEST_COMPLETE, options, + otherOptions}; + } + + @Override + protected Restorable[] restorableObjects() { + return new Restorable[0]; + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ZoneInfoTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ZoneInfoTest.java new file mode 100644 index 000000000000..923672bb85a7 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ZoneInfoTest.java @@ -0,0 +1,161 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Lists; + +import org.junit.Test; + +import java.util.LinkedList; +import java.util.List; + +public class ZoneInfoTest { + + private static final String NAME = "mz-example.com"; + private static final String ID = "123456"; + private static final Long CREATION_TIME_MILLIS = 1123468321321L; + private static final String DNS_NAME = "example.com."; + private static final String DESCRIPTION = "description for the zone"; + private static final String NAME_SERVER_SET = "some set"; + private static final String NS1 = "name server 1"; + private static final String NS2 = "name server 2"; + private static final String NS3 = "name server 3"; + private static final List NAME_SERVERS = ImmutableList.of(NS1, NS2, NS3); + private static final ZoneInfo INFO = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder() + .creationTimeMillis(CREATION_TIME_MILLIS) + .id(ID) + .nameServerSet(NAME_SERVER_SET) + .nameServers(NAME_SERVERS) + .build(); + + @Test + public void testOf() { + ZoneInfo partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION); + assertTrue(partial.nameServers().isEmpty()); + assertEquals(NAME, partial.name()); + assertNull(partial.id()); + assertNull(partial.creationTimeMillis()); + assertNull(partial.nameServerSet()); + assertEquals(DESCRIPTION, partial.description()); + assertEquals(DNS_NAME, partial.dnsName()); + } + + @Test + public void testBuilder() { + assertEquals(3, INFO.nameServers().size()); + assertEquals(NS1, INFO.nameServers().get(0)); + assertEquals(NS2, INFO.nameServers().get(1)); + assertEquals(NS3, INFO.nameServers().get(2)); + assertEquals(NAME, INFO.name()); + assertEquals(ID, INFO.id()); + assertEquals(CREATION_TIME_MILLIS, INFO.creationTimeMillis()); + assertEquals(NAME_SERVER_SET, INFO.nameServerSet()); + assertEquals(DESCRIPTION, INFO.description()); + assertEquals(DNS_NAME, INFO.dnsName()); + } + + @Test + public void testEqualsAndNotEquals() { + ZoneInfo clone = INFO.toBuilder().build(); + assertEquals(INFO, clone); + List moreServers = Lists.newLinkedList(NAME_SERVERS); + moreServers.add(NS1); + clone = INFO.toBuilder().nameServers(moreServers).build(); + assertNotEquals(INFO, clone); + String differentName = "totally different name"; + clone = INFO.toBuilder().name(differentName).build(); + assertNotEquals(INFO, clone); + clone = INFO.toBuilder().creationTimeMillis(INFO.creationTimeMillis() + 1).build(); + assertNotEquals(INFO, clone); + clone = INFO.toBuilder().description(INFO.description() + "aaaa").build(); + assertNotEquals(INFO, clone); + clone = INFO.toBuilder().dnsName(differentName).build(); + assertNotEquals(INFO, clone); + clone = INFO.toBuilder().id(INFO.id() + "1111").build(); + assertNotEquals(INFO, clone); + clone = INFO.toBuilder().nameServerSet(INFO.nameServerSet() + "salt").build(); + assertNotEquals(INFO, clone); + } + + @Test + public void testSameHashCodeOnEquals() { + int hash = INFO.hashCode(); + ZoneInfo clone = INFO.toBuilder().build(); + assertEquals(clone.hashCode(), hash); + } + + @Test + public void testToBuilder() { + assertEquals(INFO, INFO.toBuilder().build()); + ZoneInfo partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION); + assertEquals(partial, partial.toBuilder().build()); + partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder().id(ID).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder() + .creationTimeMillis(CREATION_TIME_MILLIS).build(); + assertEquals(partial, partial.toBuilder().build()); + List nameServers = new LinkedList<>(); + nameServers.add(NS1); + partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder().nameServers(nameServers).build(); + assertEquals(partial, partial.toBuilder().build()); + partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder().nameServerSet(NAME_SERVER_SET) + .build(); + assertEquals(partial, partial.toBuilder().build()); + } + + @Test + public void testToAndFromPb() { + assertEquals(INFO, ZoneInfo.fromPb(INFO.toPb())); + ZoneInfo partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION); + assertEquals(partial, ZoneInfo.fromPb(partial.toPb())); + partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder().id(ID).build(); + assertEquals(partial, ZoneInfo.fromPb(partial.toPb())); + partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder() + .creationTimeMillis(CREATION_TIME_MILLIS).build(); + assertEquals(partial, ZoneInfo.fromPb(partial.toPb())); + List nameServers = new LinkedList<>(); + nameServers.add(NS1); + partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder().nameServers(nameServers).build(); + assertEquals(partial, ZoneInfo.fromPb(partial.toPb())); + partial = ZoneInfo.of(NAME, DNS_NAME, DESCRIPTION).toBuilder().nameServerSet(NAME_SERVER_SET) + .build(); + assertEquals(partial, ZoneInfo.fromPb(partial.toPb())); + } + + @Test + public void testEmptyNameServers() { + ZoneInfo clone = INFO.toBuilder().nameServers(new LinkedList()).build(); + assertTrue(clone.nameServers().isEmpty()); + clone.toPb(); // test that this is allowed + } + + @Test + public void testDateParsing() { + com.google.api.services.dns.model.ManagedZone pb = INFO.toPb(); + pb.setCreationTime("2016-01-19T18:00:12.854Z"); // a real value obtained from Google Cloud DNS + ZoneInfo mz = ZoneInfo.fromPb(pb); // parses the string timestamp to millis + com.google.api.services.dns.model.ManagedZone pbClone = mz.toPb(); // converts it back to string + assertEquals(pb, pbClone); + assertEquals(pb.getCreationTime(), pbClone.getCreationTime()); + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ZoneTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ZoneTest.java new file mode 100644 index 000000000000..ba4493abfca8 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/ZoneTest.java @@ -0,0 +1,500 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns; + +import static org.easymock.EasyMock.createStrictMock; +import static org.easymock.EasyMock.expect; +import static org.easymock.EasyMock.replay; +import static org.easymock.EasyMock.reset; +import static org.easymock.EasyMock.verify; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertSame; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import com.google.common.collect.ImmutableList; +import com.google.gcloud.Page; + +import org.junit.After; +import org.junit.Before; +import org.junit.Test; + +import java.math.BigInteger; + +public class ZoneTest { + + private static final String ZONE_NAME = "dns-zone-name"; + private static final String ZONE_ID = "123"; + private static final ZoneInfo ZONE_INFO = Zone.of(ZONE_NAME, "example.com", "description") + .toBuilder() + .id(ZONE_ID) + .creationTimeMillis(123478946464L) + .build(); + private static final ZoneInfo NO_ID_INFO = + ZoneInfo.of(ZONE_NAME, "another-example.com", "description").toBuilder() + .creationTimeMillis(893123464L) + .build(); + private static final Dns.ZoneOption ZONE_FIELD_OPTIONS = + Dns.ZoneOption.fields(Dns.ZoneField.CREATION_TIME); + private static final Dns.RecordSetListOption DNS_RECORD_OPTIONS = + Dns.RecordSetListOption.dnsName("some-dns"); + private static final Dns.ChangeRequestOption CHANGE_REQUEST_FIELD_OPTIONS = + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.START_TIME); + private static final Dns.ChangeRequestListOption CHANGE_REQUEST_LIST_OPTIONS = + Dns.ChangeRequestListOption.fields(Dns.ChangeRequestField.START_TIME); + private static final ChangeRequestInfo CHANGE_REQUEST = + ChangeRequestInfo.builder().id("someid").build(); + private static final ChangeRequestInfo CHANGE_REQUEST_NO_ID = + ChangeRequestInfo.builder().build(); + private static final DnsException EXCEPTION = createStrictMock(DnsException.class); + private static final DnsOptions OPTIONS = createStrictMock(DnsOptions.class); + + private Dns dns; + private Zone zone; + private Zone zoneNoId; + private ChangeRequest changeRequestAfter; + + @Before + public void setUp() throws Exception { + dns = createStrictMock(Dns.class); + expect(dns.options()).andReturn(OPTIONS).times(3); + replay(dns); + zone = new Zone(dns, new ZoneInfo.BuilderImpl(ZONE_INFO)); + zoneNoId = new Zone(dns, new ZoneInfo.BuilderImpl(NO_ID_INFO)); + changeRequestAfter = new ChangeRequest(dns, ZONE_NAME, new ChangeRequestInfo.BuilderImpl( + CHANGE_REQUEST.toBuilder().startTimeMillis(123465L).build())); + reset(dns); + } + + @After + public void tearDown() throws Exception { + verify(dns); + } + + @Test + public void testConstructor() { + replay(dns); + assertEquals(ZONE_INFO.toPb(), zone.toPb()); + assertNotNull(zone.dns()); + assertEquals(dns, zone.dns()); + } + + @Test + public void deleteByNameAndFound() { + expect(dns.delete(ZONE_NAME)).andReturn(true).times(2); + replay(dns); + boolean result = zone.delete(); + assertTrue(result); + result = zoneNoId.delete(); + assertTrue(result); + } + + @Test + public void deleteByNameAndNotFound() { + expect(dns.delete(ZONE_NAME)).andReturn(false).times(2); + replay(dns); + boolean result = zoneNoId.delete(); + assertFalse(result); + result = zone.delete(); + assertFalse(result); + } + + @Test + public void listDnsRecordsByNameAndFound() { + @SuppressWarnings("unchecked") + Page pageMock = createStrictMock(Page.class); + replay(pageMock); + expect(dns.listRecordSets(ZONE_NAME)).andReturn(pageMock).times(2); + // again for options + expect(dns.listRecordSets(ZONE_NAME, DNS_RECORD_OPTIONS)).andReturn(pageMock).times(2); + replay(dns); + Page result = zone.listRecordSets(); + assertSame(pageMock, result); + result = zoneNoId.listRecordSets(); + assertSame(pageMock, result); + verify(pageMock); + zone.listRecordSets(DNS_RECORD_OPTIONS); // check options + zoneNoId.listRecordSets(DNS_RECORD_OPTIONS); // check options + } + + @Test + public void listDnsRecordsByNameAndNotFound() { + expect(dns.listRecordSets(ZONE_NAME)).andThrow(EXCEPTION).times(2); + // again for options + expect(dns.listRecordSets(ZONE_NAME, DNS_RECORD_OPTIONS)).andThrow(EXCEPTION).times(2); + replay(dns); + try { + zoneNoId.listRecordSets(); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zone.listRecordSets(); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zoneNoId.listRecordSets(DNS_RECORD_OPTIONS); // check options + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zone.listRecordSets(DNS_RECORD_OPTIONS); // check options + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + } + + @Test + public void reloadByNameAndFound() { + expect(dns.getZone(ZONE_NAME)).andReturn(zone).times(2); + // again for options + expect(dns.getZone(ZONE_NAME, ZONE_FIELD_OPTIONS)).andReturn(zoneNoId); + expect(dns.getZone(ZONE_NAME, ZONE_FIELD_OPTIONS)).andReturn(zone); + replay(dns); + Zone result = zoneNoId.reload(); + assertSame(zone.dns(), result.dns()); + assertEquals(zone, result); + result = zone.reload(); + assertSame(zone.dns(), result.dns()); + assertEquals(zone, result); + zoneNoId.reload(ZONE_FIELD_OPTIONS); // check options + zone.reload(ZONE_FIELD_OPTIONS); // check options + } + + @Test + public void reloadByNameAndNotFound() { + expect(dns.getZone(ZONE_NAME)).andReturn(null).times(2); + // again for options + expect(dns.getZone(ZONE_NAME, ZONE_FIELD_OPTIONS)).andReturn(null).times(2); + replay(dns); + Zone result = zoneNoId.reload(); + assertNull(result); + result = zone.reload(); + assertNull(result); + zoneNoId.reload(ZONE_FIELD_OPTIONS); // for options + zone.reload(ZONE_FIELD_OPTIONS); // for options + } + + @Test + public void applyChangeByNameAndFound() { + expect(dns.applyChangeRequest(ZONE_NAME, CHANGE_REQUEST)) + .andReturn(changeRequestAfter); + expect(dns.applyChangeRequest(ZONE_NAME, CHANGE_REQUEST)) + .andReturn(changeRequestAfter); + // again for options + expect(dns.applyChangeRequest(ZONE_NAME, CHANGE_REQUEST, CHANGE_REQUEST_FIELD_OPTIONS)) + .andReturn(changeRequestAfter); + expect(dns.applyChangeRequest(ZONE_NAME, CHANGE_REQUEST, CHANGE_REQUEST_FIELD_OPTIONS)) + .andReturn(changeRequestAfter); + replay(dns); + ChangeRequest result = zoneNoId.applyChangeRequest(CHANGE_REQUEST); + assertEquals(changeRequestAfter, result); + result = zone.applyChangeRequest(CHANGE_REQUEST); + assertEquals(changeRequestAfter, result); + // check options + result = zoneNoId.applyChangeRequest(CHANGE_REQUEST, CHANGE_REQUEST_FIELD_OPTIONS); + assertEquals(changeRequestAfter, result); + result = zone.applyChangeRequest(CHANGE_REQUEST, CHANGE_REQUEST_FIELD_OPTIONS); + assertEquals(changeRequestAfter, result); + } + + @Test + public void applyChangeByNameAndNotFound() { + // ID is not set + expect(dns.applyChangeRequest(ZONE_NAME, CHANGE_REQUEST)).andThrow(EXCEPTION).times(2); + // again for options + expect(dns.applyChangeRequest(ZONE_NAME, CHANGE_REQUEST, CHANGE_REQUEST_FIELD_OPTIONS)) + .andThrow(EXCEPTION).times(2); + replay(dns); + try { + zoneNoId.applyChangeRequest(CHANGE_REQUEST); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zone.applyChangeRequest(CHANGE_REQUEST); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + // check options + try { + zoneNoId.applyChangeRequest(CHANGE_REQUEST, CHANGE_REQUEST_FIELD_OPTIONS); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zone.applyChangeRequest(CHANGE_REQUEST, CHANGE_REQUEST_FIELD_OPTIONS); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + } + + @Test + public void applyNullChangeRequest() { + replay(dns); // no calls expected + try { + zone.applyChangeRequest(null); + fail("Cannot apply null ChangeRequest."); + } catch (NullPointerException e) { + // expected + } + try { + zone.applyChangeRequest(null, CHANGE_REQUEST_FIELD_OPTIONS); + fail("Cannot apply null ChangeRequest."); + } catch (NullPointerException e) { + // expected + } + try { + zoneNoId.applyChangeRequest(null); + fail("Cannot apply null ChangeRequest."); + } catch (NullPointerException e) { + // expected + } + try { + zoneNoId.applyChangeRequest(null, CHANGE_REQUEST_FIELD_OPTIONS); + fail("Cannot apply null ChangeRequest."); + } catch (NullPointerException e) { + // expected + } + } + + @Test + public void getChangeAndZoneFoundByName() { + expect(dns.getChangeRequest(ZONE_NAME, CHANGE_REQUEST.id())) + .andReturn(changeRequestAfter).times(2); + // again for options + expect(dns.getChangeRequest(ZONE_NAME, CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS)) + .andReturn(changeRequestAfter).times(2); + replay(dns); + ChangeRequest result = zoneNoId.getChangeRequest(CHANGE_REQUEST.id()); + assertEquals(changeRequestAfter, result); + result = zone.getChangeRequest(CHANGE_REQUEST.id()); + assertEquals(changeRequestAfter, result); + // check options + result = zoneNoId.getChangeRequest(CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS); + assertEquals(changeRequestAfter, result); + result = zone.getChangeRequest(CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS); + assertEquals(changeRequestAfter, result); + } + + @Test + public void getChangeAndZoneNotFoundByName() { + expect(dns.getChangeRequest(ZONE_NAME, CHANGE_REQUEST.id())).andThrow(EXCEPTION).times(2); + // again for options + expect(dns.getChangeRequest(ZONE_NAME, CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS)) + .andThrow(EXCEPTION).times(2); + replay(dns); + try { + zoneNoId.getChangeRequest(CHANGE_REQUEST.id()); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zone.getChangeRequest(CHANGE_REQUEST.id()); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + // check options + try { + zoneNoId.getChangeRequest(CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zone.getChangeRequest(CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + } + + @Test + public void getChangedWhichDoesNotExistZoneFound() { + expect(dns.getChangeRequest(ZONE_NAME, CHANGE_REQUEST.id())).andReturn(null).times(2); + // again for options + expect(dns.getChangeRequest(ZONE_NAME, CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS)) + .andReturn(null).times(2); + replay(dns); + assertNull(zoneNoId.getChangeRequest(CHANGE_REQUEST.id())); + assertNull(zone.getChangeRequest(CHANGE_REQUEST.id())); + assertNull(zoneNoId.getChangeRequest(CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS)); + assertNull(zone.getChangeRequest(CHANGE_REQUEST.id(), CHANGE_REQUEST_FIELD_OPTIONS)); + } + + @Test + public void getNullChangeRequest() { + replay(dns); // no calls expected + try { + zone.getChangeRequest(null); + fail("Cannot get null ChangeRequest."); + } catch (NullPointerException e) { + // expected + } + try { + zone.getChangeRequest(null, CHANGE_REQUEST_FIELD_OPTIONS); + fail("Cannot get null ChangeRequest."); + } catch (NullPointerException e) { + // expected + } + try { + zoneNoId.getChangeRequest(null); + fail("Cannot get null ChangeRequest."); + } catch (NullPointerException e) { + // expected + } + try { + zoneNoId.getChangeRequest(null, CHANGE_REQUEST_FIELD_OPTIONS); + fail("Cannot get null ChangeRequest."); + } catch (NullPointerException e) { + // expected + } + } + + @Test + public void getChangeRequestWithNoId() { + replay(dns); // no calls expected + try { + zone.getChangeRequest(CHANGE_REQUEST_NO_ID.id()); + fail("Cannot get ChangeRequest by null id."); + } catch (NullPointerException e) { + // expected + } + try { + zone.getChangeRequest(CHANGE_REQUEST_NO_ID.id(), CHANGE_REQUEST_FIELD_OPTIONS); + fail("Cannot get ChangeRequest by null id."); + } catch (NullPointerException e) { + // expected + } + try { + zoneNoId.getChangeRequest(CHANGE_REQUEST_NO_ID.id()); + fail("Cannot get ChangeRequest by null id."); + } catch (NullPointerException e) { + // expected + } + try { + zoneNoId.getChangeRequest(CHANGE_REQUEST_NO_ID.id(), CHANGE_REQUEST_FIELD_OPTIONS); + fail("Cannot get ChangeRequest by null id."); + } catch (NullPointerException e) { + // expected + } + } + + @Test + public void listChangeRequestsAndZoneFound() { + @SuppressWarnings("unchecked") + Page pageMock = createStrictMock(Page.class); + replay(pageMock); + expect(dns.listChangeRequests(ZONE_NAME)).andReturn(pageMock).times(2); + // again for options + expect(dns.listChangeRequests(ZONE_NAME, CHANGE_REQUEST_LIST_OPTIONS)) + .andReturn(pageMock).times(2); + replay(dns); + Page result = zoneNoId.listChangeRequests(); + assertSame(pageMock, result); + result = zone.listChangeRequests(); + assertSame(pageMock, result); + verify(pageMock); + zoneNoId.listChangeRequests(CHANGE_REQUEST_LIST_OPTIONS); // check options + zone.listChangeRequests(CHANGE_REQUEST_LIST_OPTIONS); // check options + } + + @Test + public void listChangeRequestsAndZoneNotFound() { + expect(dns.listChangeRequests(ZONE_NAME)).andThrow(EXCEPTION).times(2); + // again for options + expect(dns.listChangeRequests(ZONE_NAME, CHANGE_REQUEST_LIST_OPTIONS)).andThrow(EXCEPTION) + .times(2); + replay(dns); + try { + zoneNoId.listChangeRequests(); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zone.listChangeRequests(); + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zoneNoId.listChangeRequests(CHANGE_REQUEST_LIST_OPTIONS); // check options + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + try { + zone.listChangeRequests(CHANGE_REQUEST_LIST_OPTIONS); // check options + fail("Parent container not found, should throw an exception."); + } catch (DnsException e) { + // expected + } + } + + @Test + public void testFromPb() { + expect(dns.options()).andReturn(OPTIONS); + replay(dns); + assertEquals(Zone.fromPb(dns, zone.toPb()), zone); + } + + @Test + public void testEqualsAndToBuilder() { + expect(dns.options()).andReturn(OPTIONS).times(2); + replay(dns); + assertEquals(zone, zone.toBuilder().build()); + assertEquals(zone.hashCode(), zone.toBuilder().build().hashCode()); + } + + @Test + public void testBuilder() { + // one for each build() call because it invokes a constructor + expect(dns.options()).andReturn(OPTIONS).times(8); + replay(dns); + assertNotEquals(zone, zone.toBuilder() + .id((new BigInteger(zone.id())).add(BigInteger.ONE).toString()) + .build()); + assertNotEquals(zone, zone.toBuilder().dnsName(zone.name() + "aaaa").build()); + assertNotEquals(zone, zone.toBuilder().nameServerSet(zone.nameServerSet() + "aaaa").build()); + assertNotEquals(zone, zone.toBuilder().nameServers(ImmutableList.of("nameserverpppp")).build()); + assertNotEquals(zone, zone.toBuilder().dnsName(zone.dnsName() + "aaaa").build()); + assertNotEquals(zone, zone.toBuilder().creationTimeMillis(zone.creationTimeMillis() + 1) + .build()); + Zone.Builder builder = zone.toBuilder(); + builder.id(ZONE_ID) + .dnsName("example.com") + .creationTimeMillis(123478946464L) + .build(); + assertEquals(zone, builder.build()); + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/it/ITDnsTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/it/ITDnsTest.java new file mode 100644 index 000000000000..dd8e21043181 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/it/ITDnsTest.java @@ -0,0 +1,960 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns.it; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import com.google.common.collect.ImmutableList; +import com.google.gcloud.Page; +import com.google.gcloud.dns.ChangeRequest; +import com.google.gcloud.dns.ChangeRequestInfo; +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsException; +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.dns.ProjectInfo; +import com.google.gcloud.dns.RecordSet; +import com.google.gcloud.dns.Zone; +import com.google.gcloud.dns.ZoneInfo; + +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.Timeout; + +import java.util.Iterator; +import java.util.LinkedList; +import java.util.List; +import java.util.UUID; +import java.util.concurrent.TimeUnit; + +public class ITDnsTest { + + private static final String PREFIX = "gcldjvit-"; + private static final Dns DNS = DnsOptions.defaultInstance().service(); + private static final String ZONE_NAME1 = (PREFIX + UUID.randomUUID()).substring(0, 32); + private static final String ZONE_NAME_EMPTY_DESCRIPTION = + (PREFIX + UUID.randomUUID()).substring(0, 32); + private static final String ZONE_NAME_TOO_LONG = PREFIX + UUID.randomUUID(); + private static final String ZONE_DESCRIPTION1 = "first zone"; + private static final String ZONE_DNS_NAME1 = ZONE_NAME1 + ".com."; + private static final String ZONE_DNS_EMPTY_DESCRIPTION = ZONE_NAME_EMPTY_DESCRIPTION + ".com."; + private static final String ZONE_DNS_NAME_NO_PERIOD = ZONE_NAME1 + ".com"; + private static final ZoneInfo ZONE1 = + ZoneInfo.of(ZONE_NAME1, ZONE_DNS_EMPTY_DESCRIPTION, ZONE_DESCRIPTION1); + private static final ZoneInfo ZONE_EMPTY_DESCRIPTION = + ZoneInfo.of(ZONE_NAME_EMPTY_DESCRIPTION, ZONE_DNS_NAME1, ZONE_DESCRIPTION1); + private static final ZoneInfo ZONE_NAME_ERROR = + ZoneInfo.of(ZONE_NAME_TOO_LONG, ZONE_DNS_NAME1, ZONE_DESCRIPTION1); + private static final ZoneInfo ZONE_DNS_NO_PERIOD = + ZoneInfo.of(ZONE_NAME1, ZONE_DNS_NAME_NO_PERIOD, ZONE_DESCRIPTION1); + private static final RecordSet A_RECORD_ZONE1 = + RecordSet.builder("www." + ZONE1.dnsName(), RecordSet.Type.A) + .records(ImmutableList.of("123.123.55.1")) + .ttl(25, TimeUnit.SECONDS) + .build(); + private static final RecordSet AAAA_RECORD_ZONE1 = + RecordSet.builder("www." + ZONE1.dnsName(), RecordSet.Type.AAAA) + .records(ImmutableList.of("ed:ed:12:aa:36:3:3:105")) + .ttl(25, TimeUnit.SECONDS) + .build(); + private static final ChangeRequestInfo CHANGE_ADD_ZONE1 = ChangeRequest.builder() + .add(A_RECORD_ZONE1) + .add(AAAA_RECORD_ZONE1) + .build(); + private static final ChangeRequestInfo CHANGE_DELETE_ZONE1 = ChangeRequest.builder() + .delete(A_RECORD_ZONE1) + .delete(AAAA_RECORD_ZONE1) + .build(); + private static final List ZONE_NAMES = ImmutableList.of(ZONE_NAME1, + ZONE_NAME_EMPTY_DESCRIPTION); + + private static void clear() { + for (String zoneName : ZONE_NAMES) { + Zone zone = DNS.getZone(zoneName); + if (zone != null) { + /* We wait for all changes to complete before retrieving a list of DNS records to be + deleted. Waiting is necessary as changes potentially might create more records between + when the list has been retrieved and executing the subsequent delete operation. */ + Iterator iterator = zone.listChangeRequests().iterateAll(); + while (iterator.hasNext()) { + waitForChangeToComplete(zoneName, iterator.next().id()); + } + Iterator recordSetIterator = zone.listRecordSets().iterateAll(); + List toDelete = new LinkedList<>(); + while (recordSetIterator.hasNext()) { + RecordSet recordSet = recordSetIterator.next(); + if (!ImmutableList.of(RecordSet.Type.NS, RecordSet.Type.SOA) + .contains(recordSet.type())) { + toDelete.add(recordSet); + } + } + if (!toDelete.isEmpty()) { + ChangeRequest deletion = + zone.applyChangeRequest(ChangeRequest.builder().deletions(toDelete).build()); + waitForChangeToComplete(zone.name(), deletion.id()); + } + zone.delete(); + } + } + } + + private static List filter(Iterator iterator) { + List result = new LinkedList<>(); + while (iterator.hasNext()) { + Zone zone = iterator.next(); + if (ZONE_NAMES.contains(zone.name())) { + result.add(zone); + } + } + return result; + } + + @BeforeClass + public static void before() { + clear(); + } + + @AfterClass + public static void after() { + clear(); + } + + private static void assertEqChangesIgnoreStatus(ChangeRequest expected, ChangeRequest actual) { + assertEquals(expected.additions(), actual.additions()); + assertEquals(expected.deletions(), actual.deletions()); + assertEquals(expected.id(), actual.id()); + assertEquals(expected.startTimeMillis(), actual.startTimeMillis()); + } + + private static void waitForChangeToComplete(String zoneName, String changeId) { + while (true) { + ChangeRequest changeRequest = DNS.getChangeRequest(zoneName, changeId, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS)); + if (ChangeRequest.Status.DONE.equals(changeRequest.status())) { + return; + } + try { + Thread.sleep(500); + } catch (InterruptedException e) { + fail("Thread was interrupted while waiting for change processing."); + } + } + } + + @Rule + public Timeout globalTimeout = Timeout.seconds(300); + + @Test + public void testCreateValidZone() { + try { + Zone created = DNS.create(ZONE1); + assertEquals(ZONE1.description(), created.description()); + assertEquals(ZONE1.dnsName(), created.dnsName()); + assertEquals(ZONE1.name(), created.name()); + assertNotNull(created.creationTimeMillis()); + assertNotNull(created.nameServers()); + assertNull(created.nameServerSet()); + assertNotNull(created.id()); + Zone retrieved = DNS.getZone(ZONE1.name()); + assertEquals(created, retrieved); + created = DNS.create(ZONE_EMPTY_DESCRIPTION); + assertEquals(ZONE_EMPTY_DESCRIPTION.description(), created.description()); + assertEquals(ZONE_EMPTY_DESCRIPTION.dnsName(), created.dnsName()); + assertEquals(ZONE_EMPTY_DESCRIPTION.name(), created.name()); + assertNotNull(created.creationTimeMillis()); + assertNotNull(created.nameServers()); + assertNull(created.nameServerSet()); + assertNotNull(created.id()); + retrieved = DNS.getZone(ZONE_EMPTY_DESCRIPTION.name()); + assertEquals(created, retrieved); + } finally { + DNS.delete(ZONE1.name()); + DNS.delete(ZONE_EMPTY_DESCRIPTION.name()); + } + } + + @Test + public void testCreateZoneWithErrors() { + try { + try { + DNS.create(ZONE_NAME_ERROR); + fail("Zone name is missing a period. The service returns an error."); + } catch (DnsException ex) { + // expected + assertFalse(ex.retryable()); + } + try { + DNS.create(ZONE_DNS_NO_PERIOD); + fail("Zone name is missing a period. The service returns an error."); + } catch (DnsException ex) { + // expected + assertFalse(ex.retryable()); + } + } finally { + DNS.delete(ZONE_NAME_ERROR.name()); + DNS.delete(ZONE_DNS_NO_PERIOD.name()); + } + } + + @Test + public void testCreateZoneWithOptions() { + try { + Zone created = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.CREATION_TIME)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNotNull(created.creationTimeMillis()); + assertNull(created.description()); + assertNull(created.dnsName()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); + assertNull(created.id()); + created.delete(); + created = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.DESCRIPTION)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertEquals(ZONE1.description(), created.description()); + assertNull(created.dnsName()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); + assertNull(created.id()); + created.delete(); + created = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.DNS_NAME)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertEquals(ZONE1.dnsName(), created.dnsName()); + assertNull(created.description()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); + assertNull(created.id()); + created.delete(); + created = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.NAME)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertNull(created.description()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); + assertNull(created.id()); + created.delete(); + created = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.NAME_SERVER_SET)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertNull(created.description()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); // we did not set it + assertNull(created.id()); + created.delete(); + created = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.NAME_SERVERS)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertNull(created.description()); + assertFalse(created.nameServers().isEmpty()); + assertNull(created.nameServerSet()); + assertNull(created.id()); + created.delete(); + created = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.ZONE_ID)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertNull(created.description()); + assertNotNull(created.nameServers()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNotNull(created.id()); + created.delete(); + // combination of multiple things + created = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.ZONE_ID, + Dns.ZoneField.NAME_SERVERS, Dns.ZoneField.NAME_SERVER_SET, Dns.ZoneField.DESCRIPTION)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertEquals(ZONE1.description(), created.description()); + assertFalse(created.nameServers().isEmpty()); + assertNull(created.nameServerSet()); // we did not set it + assertNotNull(created.id()); + } finally { + DNS.delete(ZONE1.name()); + } + } + + @Test + public void testGetZone() { + try { + DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.NAME)); + Zone created = DNS.getZone(ZONE1.name(), Dns.ZoneOption.fields(Dns.ZoneField.CREATION_TIME)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNotNull(created.creationTimeMillis()); + assertNull(created.description()); + assertNull(created.dnsName()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); + assertNull(created.id()); + created = DNS.getZone(ZONE1.name(), Dns.ZoneOption.fields(Dns.ZoneField.DESCRIPTION)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertEquals(ZONE1.description(), created.description()); + assertNull(created.dnsName()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); + assertNull(created.id()); + created = DNS.getZone(ZONE1.name(), Dns.ZoneOption.fields(Dns.ZoneField.DNS_NAME)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertEquals(ZONE1.dnsName(), created.dnsName()); + assertNull(created.description()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); + assertNull(created.id()); + created = DNS.getZone(ZONE1.name(), Dns.ZoneOption.fields(Dns.ZoneField.NAME)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertNull(created.description()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); + assertNull(created.id()); + created = DNS.getZone(ZONE1.name(), Dns.ZoneOption.fields(Dns.ZoneField.NAME_SERVER_SET)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertNull(created.description()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNull(created.nameServerSet()); // we did not set it + assertNull(created.id()); + created = DNS.getZone(ZONE1.name(), Dns.ZoneOption.fields(Dns.ZoneField.NAME_SERVERS)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertNull(created.description()); + assertFalse(created.nameServers().isEmpty()); + assertNull(created.nameServerSet()); + assertNull(created.id()); + created = DNS.getZone(ZONE1.name(), Dns.ZoneOption.fields(Dns.ZoneField.ZONE_ID)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertNull(created.description()); + assertNotNull(created.nameServers()); + assertTrue(created.nameServers().isEmpty()); // never returns null + assertNotNull(created.id()); + // combination of multiple things + created = DNS.getZone(ZONE1.name(), Dns.ZoneOption.fields(Dns.ZoneField.ZONE_ID, + Dns.ZoneField.NAME_SERVERS, Dns.ZoneField.NAME_SERVER_SET, Dns.ZoneField.DESCRIPTION)); + assertEquals(ZONE1.name(), created.name()); // always returned + assertNull(created.creationTimeMillis()); + assertNull(created.dnsName()); + assertEquals(ZONE1.description(), created.description()); + assertFalse(created.nameServers().isEmpty()); + assertNull(created.nameServerSet()); // we did not set it + assertNotNull(created.id()); + } finally { + DNS.delete(ZONE1.name()); + } + } + + @Test + public void testListZones() { + try { + List zones = filter(DNS.listZones().iterateAll()); + assertEquals(0, zones.size()); + // some zones exists + Zone created = DNS.create(ZONE1); + zones = filter(DNS.listZones().iterateAll()); + assertEquals(created, zones.get(0)); + assertEquals(1, zones.size()); + created = DNS.create(ZONE_EMPTY_DESCRIPTION); + zones = filter(DNS.listZones().iterateAll()); + assertEquals(2, zones.size()); + assertTrue(zones.contains(created)); + // error in options + try { + DNS.listZones(Dns.ZoneListOption.pageSize(0)); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertFalse(ex.retryable()); + } + try { + DNS.listZones(Dns.ZoneListOption.pageSize(-1)); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertFalse(ex.retryable()); + } + // ok size + zones = filter(DNS.listZones(Dns.ZoneListOption.pageSize(1000)).iterateAll()); + assertEquals(2, zones.size()); // we still have only 2 zones + // dns name problems + try { + DNS.listZones(Dns.ZoneListOption.dnsName("aaaaa")); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertFalse(ex.retryable()); + } + // ok name + zones = filter(DNS.listZones(Dns.ZoneListOption.dnsName(ZONE1.dnsName())).iterateAll()); + assertEquals(1, zones.size()); + // field options + Iterator zoneIterator = DNS.listZones(Dns.ZoneListOption.dnsName(ZONE1.dnsName()), + Dns.ZoneListOption.fields(Dns.ZoneField.ZONE_ID)).iterateAll(); + Zone zone = zoneIterator.next(); + assertNull(zone.creationTimeMillis()); + assertNotNull(zone.name()); + assertNull(zone.dnsName()); + assertNull(zone.description()); + assertNull(zone.nameServerSet()); + assertTrue(zone.nameServers().isEmpty()); + assertNotNull(zone.id()); + assertFalse(zoneIterator.hasNext()); + zoneIterator = DNS.listZones(Dns.ZoneListOption.dnsName(ZONE1.dnsName()), + Dns.ZoneListOption.fields(Dns.ZoneField.CREATION_TIME)).iterateAll(); + zone = zoneIterator.next(); + assertNotNull(zone.creationTimeMillis()); + assertNotNull(zone.name()); + assertNull(zone.dnsName()); + assertNull(zone.description()); + assertNull(zone.nameServerSet()); + assertTrue(zone.nameServers().isEmpty()); + assertNull(zone.id()); + assertFalse(zoneIterator.hasNext()); + zoneIterator = DNS.listZones(Dns.ZoneListOption.dnsName(ZONE1.dnsName()), + Dns.ZoneListOption.fields(Dns.ZoneField.DNS_NAME)).iterateAll(); + zone = zoneIterator.next(); + assertNull(zone.creationTimeMillis()); + assertNotNull(zone.name()); + assertNotNull(zone.dnsName()); + assertNull(zone.description()); + assertNull(zone.nameServerSet()); + assertTrue(zone.nameServers().isEmpty()); + assertNull(zone.id()); + assertFalse(zoneIterator.hasNext()); + zoneIterator = DNS.listZones(Dns.ZoneListOption.dnsName(ZONE1.dnsName()), + Dns.ZoneListOption.fields(Dns.ZoneField.DESCRIPTION)).iterateAll(); + zone = zoneIterator.next(); + assertNull(zone.creationTimeMillis()); + assertNotNull(zone.name()); + assertNull(zone.dnsName()); + assertNotNull(zone.description()); + assertNull(zone.nameServerSet()); + assertTrue(zone.nameServers().isEmpty()); + assertNull(zone.id()); + assertFalse(zoneIterator.hasNext()); + zoneIterator = DNS.listZones(Dns.ZoneListOption.dnsName(ZONE1.dnsName()), + Dns.ZoneListOption.fields(Dns.ZoneField.NAME_SERVERS)).iterateAll(); + zone = zoneIterator.next(); + assertNull(zone.creationTimeMillis()); + assertNotNull(zone.name()); + assertNull(zone.dnsName()); + assertNull(zone.description()); + assertNull(zone.nameServerSet()); + assertTrue(!zone.nameServers().isEmpty()); + assertNull(zone.id()); + assertFalse(zoneIterator.hasNext()); + zoneIterator = DNS.listZones(Dns.ZoneListOption.dnsName(ZONE1.dnsName()), + Dns.ZoneListOption.fields(Dns.ZoneField.NAME_SERVER_SET)).iterateAll(); + zone = zoneIterator.next(); + assertNull(zone.creationTimeMillis()); + assertNotNull(zone.name()); + assertNull(zone.dnsName()); + assertNull(zone.description()); + assertNull(zone.nameServerSet()); // we cannot set it using gcloud java + assertTrue(zone.nameServers().isEmpty()); + assertNull(zone.id()); + assertFalse(zoneIterator.hasNext()); + // several combined + zones = filter(DNS.listZones(Dns.ZoneListOption.fields(Dns.ZoneField.ZONE_ID, + Dns.ZoneField.DESCRIPTION), + Dns.ZoneListOption.pageSize(1)).iterateAll()); + assertEquals(2, zones.size()); + for (Zone current : zones) { + assertNull(current.creationTimeMillis()); + assertNotNull(current.name()); + assertNull(current.dnsName()); + assertNotNull(current.description()); + assertNull(current.nameServerSet()); + assertTrue(zone.nameServers().isEmpty()); + assertNotNull(current.id()); + } + } finally { + DNS.delete(ZONE1.name()); + DNS.delete(ZONE_EMPTY_DESCRIPTION.name()); + } + } + + @Test + public void testDeleteZone() { + try { + Zone created = DNS.create(ZONE1); + assertEquals(created, DNS.getZone(ZONE1.name())); + DNS.delete(ZONE1.name()); + assertNull(DNS.getZone(ZONE1.name())); + } finally { + DNS.delete(ZONE1.name()); + } + } + + @Test + public void testCreateChange() { + try { + DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.NAME)); + ChangeRequest created = DNS.applyChangeRequest(ZONE1.name(), CHANGE_ADD_ZONE1); + assertEquals(CHANGE_ADD_ZONE1.additions(), created.additions()); + assertNotNull(created.startTimeMillis()); + assertTrue(created.deletions().isEmpty()); + assertEquals("1", created.id()); + assertTrue(ImmutableList.of(ChangeRequest.Status.PENDING, ChangeRequest.Status.DONE) + .contains(created.status())); + assertEqChangesIgnoreStatus(created, DNS.getChangeRequest(ZONE1.name(), "1")); + waitForChangeToComplete(ZONE1.name(), "1"); + DNS.applyChangeRequest(ZONE1.name(), CHANGE_DELETE_ZONE1); + waitForChangeToComplete(ZONE1.name(), "2"); + // with options + created = DNS.applyChangeRequest(ZONE1.name(), CHANGE_ADD_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.ID)); + assertTrue(created.additions().isEmpty()); + assertNull(created.startTimeMillis()); + assertTrue(created.deletions().isEmpty()); + assertEquals("3", created.id()); + assertNull(created.status()); + waitForChangeToComplete(ZONE1.name(), "3"); + DNS.applyChangeRequest(ZONE1.name(), CHANGE_DELETE_ZONE1); + waitForChangeToComplete(ZONE1.name(), "4"); + created = DNS.applyChangeRequest(ZONE1.name(), CHANGE_ADD_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS)); + assertTrue(created.additions().isEmpty()); + assertNull(created.startTimeMillis()); + assertTrue(created.deletions().isEmpty()); + assertEquals("5", created.id()); + assertNotNull(created.status()); + waitForChangeToComplete(ZONE1.name(), "5"); + DNS.applyChangeRequest(ZONE1.name(), CHANGE_DELETE_ZONE1); + waitForChangeToComplete(ZONE1.name(), "6"); + created = DNS.applyChangeRequest(ZONE1.name(), CHANGE_ADD_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.START_TIME)); + assertTrue(created.additions().isEmpty()); + assertNotNull(created.startTimeMillis()); + assertTrue(created.deletions().isEmpty()); + assertEquals("7", created.id()); + assertNull(created.status()); + waitForChangeToComplete(ZONE1.name(), "7"); + DNS.applyChangeRequest(ZONE1.name(), CHANGE_DELETE_ZONE1); + waitForChangeToComplete(ZONE1.name(), "8"); + created = DNS.applyChangeRequest(ZONE1.name(), CHANGE_ADD_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.ADDITIONS)); + assertEquals(CHANGE_ADD_ZONE1.additions(), created.additions()); + assertNull(created.startTimeMillis()); + assertTrue(created.deletions().isEmpty()); + assertEquals("9", created.id()); + assertNull(created.status()); + // finishes with delete otherwise we cannot delete the zone + waitForChangeToComplete(ZONE1.name(), "9"); + created = DNS.applyChangeRequest(ZONE1.name(), CHANGE_DELETE_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.DELETIONS)); + waitForChangeToComplete(ZONE1.name(), "10"); + assertEquals(CHANGE_DELETE_ZONE1.deletions(), created.deletions()); + assertNull(created.startTimeMillis()); + assertTrue(created.additions().isEmpty()); + assertEquals("10", created.id()); + assertNull(created.status()); + waitForChangeToComplete(ZONE1.name(), "10"); + } finally { + clear(); + } + } + + @Test + public void testInvalidChangeRequest() { + Zone zone = DNS.create(ZONE1); + RecordSet validA = + RecordSet.builder("subdomain." + zone.dnsName(), RecordSet.Type.A) + .records(ImmutableList.of("0.255.1.5")) + .build(); + try { + ChangeRequestInfo validChange = ChangeRequest.builder().add(validA).build(); + zone.applyChangeRequest(validChange); + try { + zone.applyChangeRequest(validChange); + fail("Created a record set which already exists."); + } catch (DnsException ex) { + // expected + assertFalse(ex.retryable()); + assertEquals(409, ex.code()); + } + // delete with field mismatch + RecordSet mismatch = validA.toBuilder().ttl(20, TimeUnit.SECONDS).build(); + ChangeRequestInfo deletion = ChangeRequest.builder().delete(mismatch).build(); + try { + zone.applyChangeRequest(deletion); + fail("Deleted a record set without a complete match."); + } catch (DnsException ex) { + // expected + assertEquals(412, ex.code()); + assertFalse(ex.retryable()); + } + // delete and add SOA + Iterator recordSetIterator = zone.listRecordSets().iterateAll(); + LinkedList deletions = new LinkedList<>(); + LinkedList additions = new LinkedList<>(); + while (recordSetIterator.hasNext()) { + RecordSet recordSet = recordSetIterator.next(); + if (recordSet.type() == RecordSet.Type.SOA) { + deletions.add(recordSet); + // the subdomain is necessary to get 400 instead of 412 + RecordSet copy = recordSet.toBuilder().name("x." + recordSet.name()).build(); + additions.add(copy); + break; + } + } + deletion = deletion.toBuilder().deletions(deletions).build(); + ChangeRequestInfo addition = ChangeRequest.builder().additions(additions).build(); + try { + zone.applyChangeRequest(deletion); + fail("Deleted SOA."); + } catch (DnsException ex) { + // expected + assertFalse(ex.retryable()); + assertEquals(400, ex.code()); + } + try { + zone.applyChangeRequest(addition); + fail("Added second SOA."); + } catch (DnsException ex) { + // expected + assertFalse(ex.retryable()); + assertEquals(400, ex.code()); + } + } finally { + ChangeRequestInfo deletion = ChangeRequest.builder().delete(validA).build(); + ChangeRequest request = zone.applyChangeRequest(deletion); + waitForChangeToComplete(zone.name(), request.id()); + zone.delete(); + } + } + + @Test + public void testListChanges() { + try { + // no such zone exists + try { + DNS.listChangeRequests(ZONE1.name()); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(404, ex.code()); + assertFalse(ex.retryable()); + } + // zone exists but has no changes + DNS.create(ZONE1); + ImmutableList changes = ImmutableList.copyOf( + DNS.listChangeRequests(ZONE1.name()).iterateAll()); + assertEquals(1, changes.size()); // default change creating SOA and NS + // zone has changes + ChangeRequest change = DNS.applyChangeRequest(ZONE1.name(), CHANGE_ADD_ZONE1); + waitForChangeToComplete(ZONE1.name(), change.id()); + change = DNS.applyChangeRequest(ZONE1.name(), CHANGE_DELETE_ZONE1); + waitForChangeToComplete(ZONE1.name(), change.id()); + change = DNS.applyChangeRequest(ZONE1.name(), CHANGE_ADD_ZONE1); + waitForChangeToComplete(ZONE1.name(), change.id()); + change = DNS.applyChangeRequest(ZONE1.name(), CHANGE_DELETE_ZONE1); + waitForChangeToComplete(ZONE1.name(), change.id()); + changes = ImmutableList.copyOf(DNS.listChangeRequests(ZONE1.name()).iterateAll()); + assertEquals(5, changes.size()); + // error in options + try { + DNS.listChangeRequests(ZONE1.name(), Dns.ChangeRequestListOption.pageSize(0)); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertFalse(ex.retryable()); + } + try { + DNS.listChangeRequests(ZONE1.name(), Dns.ChangeRequestListOption.pageSize(-1)); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertFalse(ex.retryable()); + } + // sorting order + ImmutableList ascending = ImmutableList.copyOf(DNS.listChangeRequests( + ZONE1.name(), + Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.ASCENDING)).iterateAll()); + ImmutableList descending = ImmutableList.copyOf(DNS.listChangeRequests( + ZONE1.name(), + Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.DESCENDING)).iterateAll()); + int size = 5; + assertEquals(size, descending.size()); + assertEquals(size, ascending.size()); + for (int i = 0; i < size; i++) { + assertEquals(descending.get(i), ascending.get(size - i - 1)); + } + // field options + changes = ImmutableList.copyOf(DNS.listChangeRequests(ZONE1.name(), + Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.ASCENDING), + Dns.ChangeRequestListOption.fields(Dns.ChangeRequestField.ADDITIONS)).iterateAll()); + change = changes.get(1); + assertEquals(CHANGE_ADD_ZONE1.additions(), change.additions()); + assertTrue(change.deletions().isEmpty()); + assertEquals("1", change.id()); + assertNull(change.startTimeMillis()); + assertNull(change.status()); + changes = ImmutableList.copyOf(DNS.listChangeRequests(ZONE1.name(), + Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.ASCENDING), + Dns.ChangeRequestListOption.fields(Dns.ChangeRequestField.DELETIONS)).iterateAll()); + change = changes.get(2); + assertTrue(change.additions().isEmpty()); + assertNotNull(change.deletions()); + assertEquals("2", change.id()); + assertNull(change.startTimeMillis()); + assertNull(change.status()); + changes = ImmutableList.copyOf(DNS.listChangeRequests(ZONE1.name(), + Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.ASCENDING), + Dns.ChangeRequestListOption.fields(Dns.ChangeRequestField.ID)).iterateAll()); + change = changes.get(1); + assertTrue(change.additions().isEmpty()); + assertTrue(change.deletions().isEmpty()); + assertEquals("1", change.id()); + assertNull(change.startTimeMillis()); + assertNull(change.status()); + changes = ImmutableList.copyOf(DNS.listChangeRequests(ZONE1.name(), + Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.ASCENDING), + Dns.ChangeRequestListOption.fields(Dns.ChangeRequestField.START_TIME)).iterateAll()); + change = changes.get(1); + assertTrue(change.additions().isEmpty()); + assertTrue(change.deletions().isEmpty()); + assertEquals("1", change.id()); + assertNotNull(change.startTimeMillis()); + assertNull(change.status()); + changes = ImmutableList.copyOf(DNS.listChangeRequests(ZONE1.name(), + Dns.ChangeRequestListOption.sortOrder(Dns.SortingOrder.ASCENDING), + Dns.ChangeRequestListOption.fields(Dns.ChangeRequestField.STATUS)).iterateAll()); + change = changes.get(1); + assertTrue(change.additions().isEmpty()); + assertTrue(change.deletions().isEmpty()); + assertEquals("1", change.id()); + assertNull(change.startTimeMillis()); + assertEquals(ChangeRequest.Status.DONE, change.status()); + } finally { + clear(); + } + } + + @Test + public void testGetChange() { + try { + Zone zone = DNS.create(ZONE1, Dns.ZoneOption.fields(Dns.ZoneField.NAME)); + ChangeRequest created = zone.applyChangeRequest(CHANGE_ADD_ZONE1); + ChangeRequest retrieved = DNS.getChangeRequest(zone.name(), created.id()); + assertEqChangesIgnoreStatus(created, retrieved); + waitForChangeToComplete(zone.name(), created.id()); + zone.applyChangeRequest(CHANGE_DELETE_ZONE1); + // with options + created = zone.applyChangeRequest(CHANGE_ADD_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.ID)); + retrieved = DNS.getChangeRequest(zone.name(), created.id(), + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.ID)); + assertEqChangesIgnoreStatus(created, retrieved); + waitForChangeToComplete(zone.name(), created.id()); + zone.applyChangeRequest(CHANGE_DELETE_ZONE1); + created = zone.applyChangeRequest(CHANGE_ADD_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS)); + retrieved = DNS.getChangeRequest(zone.name(), created.id(), + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS)); + assertEqChangesIgnoreStatus(created, retrieved); + waitForChangeToComplete(zone.name(), created.id()); + zone.applyChangeRequest(CHANGE_DELETE_ZONE1); + created = zone.applyChangeRequest(CHANGE_ADD_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.START_TIME)); + retrieved = DNS.getChangeRequest(zone.name(), created.id(), + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.START_TIME)); + assertEqChangesIgnoreStatus(created, retrieved); + waitForChangeToComplete(zone.name(), created.id()); + zone.applyChangeRequest(CHANGE_DELETE_ZONE1); + created = zone.applyChangeRequest(CHANGE_ADD_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.ADDITIONS)); + retrieved = DNS.getChangeRequest(zone.name(), created.id(), + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.ADDITIONS)); + assertEqChangesIgnoreStatus(created, retrieved); + waitForChangeToComplete(zone.name(), created.id()); + // finishes with delete otherwise we cannot delete the zone + created = zone.applyChangeRequest(CHANGE_DELETE_ZONE1, + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.DELETIONS)); + retrieved = DNS.getChangeRequest(zone.name(), created.id(), + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.DELETIONS)); + assertEqChangesIgnoreStatus(created, retrieved); + waitForChangeToComplete(zone.name(), created.id()); + } finally { + clear(); + } + } + + @Test + public void testGetProject() { + // fetches all fields + ProjectInfo project = DNS.getProject(); + assertNotNull(project.quota()); + // options + project = DNS.getProject(Dns.ProjectOption.fields(Dns.ProjectField.QUOTA)); + assertNotNull(project.quota()); + project = DNS.getProject(Dns.ProjectOption.fields(Dns.ProjectField.PROJECT_ID)); + assertNull(project.quota()); + project = DNS.getProject(Dns.ProjectOption.fields(Dns.ProjectField.PROJECT_NUMBER)); + assertNull(project.quota()); + project = DNS.getProject(Dns.ProjectOption.fields(Dns.ProjectField.PROJECT_NUMBER, + Dns.ProjectField.QUOTA, Dns.ProjectField.PROJECT_ID)); + assertNotNull(project.quota()); + } + + @Test + public void testListDnsRecords() { + try { + Zone zone = DNS.create(ZONE1); + ImmutableList recordSets = ImmutableList.copyOf( + DNS.listRecordSets(zone.name()).iterateAll()); + assertEquals(2, recordSets.size()); + ImmutableList defaultRecords = + ImmutableList.of(RecordSet.Type.NS, RecordSet.Type.SOA); + for (RecordSet recordSet : recordSets) { + assertTrue(defaultRecords.contains(recordSet.type())); + } + // field options + Iterator recordSetIterator = DNS.listRecordSets(zone.name(), + Dns.RecordSetListOption.fields(Dns.RecordSetField.TTL)).iterateAll(); + int counter = 0; + while (recordSetIterator.hasNext()) { + RecordSet recordSet = recordSetIterator.next(); + assertEquals(recordSets.get(counter).ttl(), recordSet.ttl()); + assertEquals(recordSets.get(counter).name(), recordSet.name()); + assertEquals(recordSets.get(counter).type(), recordSet.type()); + assertTrue(recordSet.records().isEmpty()); + counter++; + } + assertEquals(2, counter); + recordSetIterator = DNS.listRecordSets(zone.name(), + Dns.RecordSetListOption.fields(Dns.RecordSetField.NAME)).iterateAll(); + counter = 0; + while (recordSetIterator.hasNext()) { + RecordSet recordSet = recordSetIterator.next(); + assertEquals(recordSets.get(counter).name(), recordSet.name()); + assertEquals(recordSets.get(counter).type(), recordSet.type()); + assertTrue(recordSet.records().isEmpty()); + assertNull(recordSet.ttl()); + counter++; + } + assertEquals(2, counter); + recordSetIterator = DNS.listRecordSets(zone.name(), + Dns.RecordSetListOption.fields(Dns.RecordSetField.DNS_RECORDS)) + .iterateAll(); + counter = 0; + while (recordSetIterator.hasNext()) { + RecordSet recordSet = recordSetIterator.next(); + assertEquals(recordSets.get(counter).records(), recordSet.records()); + assertEquals(recordSets.get(counter).name(), recordSet.name()); + assertEquals(recordSets.get(counter).type(), recordSet.type()); + assertNull(recordSet.ttl()); + counter++; + } + assertEquals(2, counter); + recordSetIterator = DNS.listRecordSets(zone.name(), + Dns.RecordSetListOption.fields(Dns.RecordSetField.TYPE), + Dns.RecordSetListOption.pageSize(1)).iterateAll(); // also test paging + counter = 0; + while (recordSetIterator.hasNext()) { + RecordSet recordSet = recordSetIterator.next(); + assertEquals(recordSets.get(counter).type(), recordSet.type()); + assertEquals(recordSets.get(counter).name(), recordSet.name()); + assertTrue(recordSet.records().isEmpty()); + assertNull(recordSet.ttl()); + counter++; + } + assertEquals(2, counter); + // test page size + Page recordSetPage = DNS.listRecordSets(zone.name(), + Dns.RecordSetListOption.fields(Dns.RecordSetField.TYPE), + Dns.RecordSetListOption.pageSize(1)); + assertEquals(1, ImmutableList.copyOf(recordSetPage.values().iterator()).size()); + // test name filter + ChangeRequest change = DNS.applyChangeRequest(ZONE1.name(), CHANGE_ADD_ZONE1); + waitForChangeToComplete(ZONE1.name(), change.id()); + recordSetIterator = DNS.listRecordSets(ZONE1.name(), + Dns.RecordSetListOption.dnsName(A_RECORD_ZONE1.name())).iterateAll(); + counter = 0; + while (recordSetIterator.hasNext()) { + RecordSet recordSet = recordSetIterator.next(); + assertTrue(ImmutableList.of(A_RECORD_ZONE1.type(), AAAA_RECORD_ZONE1.type()) + .contains(recordSet.type())); + counter++; + } + assertEquals(2, counter); + // test type filter + waitForChangeToComplete(ZONE1.name(), change.id()); + recordSetIterator = DNS.listRecordSets(ZONE1.name(), + Dns.RecordSetListOption.dnsName(A_RECORD_ZONE1.name()), + Dns.RecordSetListOption.type(A_RECORD_ZONE1.type())) + .iterateAll(); + counter = 0; + while (recordSetIterator.hasNext()) { + RecordSet recordSet = recordSetIterator.next(); + assertEquals(A_RECORD_ZONE1, recordSet); + counter++; + } + assertEquals(1, counter); + change = zone.applyChangeRequest(CHANGE_DELETE_ZONE1); + // check wrong arguments + try { + // name is not set + DNS.listRecordSets(ZONE1.name(), + Dns.RecordSetListOption.type(A_RECORD_ZONE1.type())); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertFalse(ex.retryable()); + } + try { + DNS.listRecordSets(ZONE1.name(), Dns.RecordSetListOption.pageSize(0)); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertFalse(ex.retryable()); + } + try { + DNS.listRecordSets(ZONE1.name(), Dns.RecordSetListOption.pageSize(-1)); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertFalse(ex.retryable()); + } + waitForChangeToComplete(ZONE1.name(), change.id()); + } finally { + clear(); + } + } +} diff --git a/gcloud-java-dns/src/test/java/com/google/gcloud/dns/testing/LocalDnsHelperTest.java b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/testing/LocalDnsHelperTest.java new file mode 100644 index 000000000000..44516f47c657 --- /dev/null +++ b/gcloud-java-dns/src/test/java/com/google/gcloud/dns/testing/LocalDnsHelperTest.java @@ -0,0 +1,1455 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.dns.testing; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +import com.google.api.services.dns.model.Change; +import com.google.api.services.dns.model.ManagedZone; +import com.google.api.services.dns.model.Project; +import com.google.api.services.dns.model.ResourceRecordSet; +import com.google.common.collect.ImmutableCollection; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Lists; +import com.google.gcloud.dns.DnsException; +import com.google.gcloud.dns.spi.DefaultDnsRpc; +import com.google.gcloud.dns.spi.DnsRpc; + +import org.junit.AfterClass; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.Timeout; + +import java.util.HashMap; +import java.util.HashSet; +import java.util.Iterator; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; + +public class LocalDnsHelperTest { + + private static final String RRSET_TYPE = "A"; + private static final ResourceRecordSet RRSET1 = new ResourceRecordSet(); + private static final ResourceRecordSet RRSET2 = new ResourceRecordSet(); + private static final ResourceRecordSet RRSET_KEEP = new ResourceRecordSet(); + private static final String PROJECT_ID1 = "2135436541254"; + private static final String PROJECT_ID2 = "882248761325"; + private static final String ZONE_NAME1 = "my-little-zone"; + private static final String ZONE_NAME2 = "another-zone-name"; + private static final ManagedZone ZONE1 = new ManagedZone(); + private static final ManagedZone ZONE2 = new ManagedZone(); + private static final String DNS_NAME = "www.example.com."; + private static final Change CHANGE1 = new Change(); + private static final Change CHANGE2 = new Change(); + private static final Change CHANGE_KEEP = new Change(); + private static final Change CHANGE_COMPLEX = new Change(); + private static final LocalDnsHelper LOCAL_DNS_HELPER = LocalDnsHelper.create(0L); + private static final Map EMPTY_RPC_OPTIONS = ImmutableMap.of(); + private static final DnsRpc RPC = new DefaultDnsRpc(LOCAL_DNS_HELPER.options()); + private static final String REAL_PROJECT_ID = LOCAL_DNS_HELPER.options().projectId(); + private Map optionsMap; + + @BeforeClass + public static void before() { + ZONE1.setName(ZONE_NAME1); + ZONE1.setDescription(""); + ZONE1.setDnsName(DNS_NAME); + ZONE1.setNameServerSet("somenameserverset"); + ZONE2.setName(ZONE_NAME2); + ZONE2.setDescription(""); + ZONE2.setDnsName(DNS_NAME); + ZONE2.setNameServerSet("somenameserverset"); + RRSET1.setName(DNS_NAME); + RRSET1.setType(RRSET_TYPE); + RRSET1.setRrdatas(ImmutableList.of("1.1.1.1")); + RRSET2.setName(DNS_NAME); + RRSET2.setType(RRSET_TYPE); + RRSET2.setRrdatas(ImmutableList.of("123.132.153.156")); + RRSET_KEEP.setName(DNS_NAME); + RRSET_KEEP.setType("MX"); + RRSET_KEEP.setRrdatas(ImmutableList.of("255.255.255.254")); + CHANGE1.setAdditions(ImmutableList.of(RRSET1, RRSET2)); + CHANGE2.setDeletions(ImmutableList.of(RRSET2)); + CHANGE_KEEP.setAdditions(ImmutableList.of(RRSET_KEEP)); + CHANGE_COMPLEX.setAdditions(ImmutableList.of(RRSET_KEEP)); + CHANGE_COMPLEX.setDeletions(ImmutableList.of(RRSET_KEEP)); + LOCAL_DNS_HELPER.start(); + } + + @Rule + public Timeout globalTimeout = Timeout.seconds(60); + + @Before + public void setUp() { + resetProjects(); + optionsMap = new HashMap<>(); + } + + @AfterClass + public static void after() { + LOCAL_DNS_HELPER.stop(); + } + + private static void resetProjects() { + for (String project : LOCAL_DNS_HELPER.projects().keySet()) { + LOCAL_DNS_HELPER.projects().remove(project); + } + } + + private static void assertEqChangesIgnoreStatus(Change expected, Change actual) { + assertEquals(expected.getAdditions(), actual.getAdditions()); + assertEquals(expected.getDeletions(), actual.getDeletions()); + assertEquals(expected.getId(), actual.getId()); + assertEquals(expected.getStartTime(), actual.getStartTime()); + } + + @Test + public void testCreateZone() { + ManagedZone created = RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + // check that default records were created + DnsRpc.ListResult listResult + = RPC.listRecordSets(ZONE1.getName(), EMPTY_RPC_OPTIONS); + ImmutableList defaultTypes = ImmutableList.of("SOA", "NS"); + Iterator iterator = listResult.results().iterator(); + assertTrue(defaultTypes.contains(iterator.next().getType())); + assertTrue(defaultTypes.contains(iterator.next().getType())); + assertFalse(iterator.hasNext()); + assertEquals(created, LOCAL_DNS_HELPER.findZone(REAL_PROJECT_ID, ZONE1.getName()).zone()); + ManagedZone zone = RPC.getZone(ZONE_NAME1, EMPTY_RPC_OPTIONS); + assertEquals(created, zone); + try { + RPC.create(null, EMPTY_RPC_OPTIONS); + fail("Zone cannot be null"); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("entity.managedZone")); + } + // create zone twice + try { + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + fail("Zone already exists."); + } catch (DnsException ex) { + // expected + assertEquals(409, ex.code()); + assertTrue(ex.getMessage().contains("already exists")); + } + // field options + resetProjects(); + Map options = new HashMap<>(); + options.put(DnsRpc.Option.FIELDS, "id"); + zone = RPC.create(ZONE1, options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNotNull(zone.getId()); + resetProjects(); + options.put(DnsRpc.Option.FIELDS, "creationTime"); + zone = RPC.create(ZONE1, options); + assertNotNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "dnsName"); + resetProjects(); + zone = RPC.create(ZONE1, options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNotNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "description"); + resetProjects(); + zone = RPC.create(ZONE1, options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNotNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "nameServers"); + resetProjects(); + zone = RPC.create(ZONE1, options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNotNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "nameServerSet"); + resetProjects(); + zone = RPC.create(ZONE1, options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNotNull(zone.getNameServerSet()); + assertNull(zone.getId()); + // several combined + options.put(DnsRpc.Option.FIELDS, "nameServerSet,description,id,name"); + resetProjects(); + zone = RPC.create(ZONE1, options); + assertNull(zone.getCreationTime()); + assertNotNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNotNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNotNull(zone.getNameServerSet()); + assertNotNull(zone.getId()); + } + + @Test + public void testGetZone() { + // non-existent + assertNull(RPC.getZone(ZONE_NAME1, EMPTY_RPC_OPTIONS)); + // existent + ManagedZone created = RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + ManagedZone zone = RPC.getZone(ZONE_NAME1, EMPTY_RPC_OPTIONS); + assertEquals(created, zone); + assertEquals(ZONE1.getName(), zone.getName()); + // field options + Map options = new HashMap<>(); + options.put(DnsRpc.Option.FIELDS, "id"); + zone = RPC.getZone(ZONE1.getName(), options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNotNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "creationTime"); + zone = RPC.getZone(ZONE1.getName(), options); + assertNotNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "dnsName"); + zone = RPC.getZone(ZONE1.getName(), options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNotNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "description"); + zone = RPC.getZone(ZONE1.getName(), options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNotNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "nameServers"); + zone = RPC.getZone(ZONE1.getName(), options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNotNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "nameServerSet"); + zone = RPC.getZone(ZONE1.getName(), options); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNotNull(zone.getNameServerSet()); + assertNull(zone.getId()); + // several combined + options.put(DnsRpc.Option.FIELDS, "nameServerSet,description,id,name"); + zone = RPC.getZone(ZONE1.getName(), options); + assertNull(zone.getCreationTime()); + assertNotNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNotNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNotNull(zone.getNameServerSet()); + assertNotNull(zone.getId()); + } + + @Test + public void testDeleteZone() { + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + assertTrue(RPC.deleteZone(ZONE1.getName())); + assertNull(RPC.getZone(ZONE1.getName(), EMPTY_RPC_OPTIONS)); + // deleting non-existent zone + assertFalse(RPC.deleteZone(ZONE1.getName())); + assertNull(RPC.getZone(ZONE1.getName(), EMPTY_RPC_OPTIONS)); + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + RPC.create(ZONE2, EMPTY_RPC_OPTIONS); + assertNotNull(RPC.getZone(ZONE1.getName(), EMPTY_RPC_OPTIONS)); + assertNotNull(RPC.getZone(ZONE2.getName(), EMPTY_RPC_OPTIONS)); + // delete in reverse order + assertTrue(RPC.deleteZone(ZONE1.getName())); + assertNull(RPC.getZone(ZONE1.getName(), EMPTY_RPC_OPTIONS)); + assertNotNull(RPC.getZone(ZONE2.getName(), EMPTY_RPC_OPTIONS)); + assertTrue(RPC.deleteZone(ZONE2.getName())); + assertNull(RPC.getZone(ZONE1.getName(), EMPTY_RPC_OPTIONS)); + assertNull(RPC.getZone(ZONE2.getName(), EMPTY_RPC_OPTIONS)); + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + RPC.applyChangeRequest(ZONE1.getName(), CHANGE_KEEP, EMPTY_RPC_OPTIONS); + try { + RPC.deleteZone(ZONE1.getName()); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("not empty")); + } + } + + @Test + public void testCreateAndApplyChange() { + executeCreateAndApplyChangeTest(RPC); + } + + @Test + public void testCreateAndApplyChangeWithThreads() { + LocalDnsHelper localDnsThreaded = LocalDnsHelper.create(50L); + localDnsThreaded.start(); + DnsRpc rpc = new DefaultDnsRpc(localDnsThreaded.options()); + executeCreateAndApplyChangeTest(rpc); + localDnsThreaded.stop(); + } + + private static void waitForChangeToComplete(DnsRpc rpc, String zoneName, String changeId) { + while (true) { + Change change = rpc.getChangeRequest(zoneName, changeId, EMPTY_RPC_OPTIONS); + if ("done".equals(change.getStatus())) { + return; + } + try { + Thread.sleep(50L); + } catch (InterruptedException e) { + fail("Thread was interrupted while waiting for change processing."); + } + } + } + + private static void executeCreateAndApplyChangeTest(DnsRpc rpc) { + rpc.create(ZONE1, EMPTY_RPC_OPTIONS); + assertNull(rpc.getChangeRequest(ZONE1.getName(), "1", EMPTY_RPC_OPTIONS)); + // add + Change createdChange = rpc.applyChangeRequest(ZONE1.getName(), CHANGE1, EMPTY_RPC_OPTIONS); + assertEquals(CHANGE1.getAdditions(), createdChange.getAdditions()); + assertEquals(CHANGE1.getDeletions(), createdChange.getDeletions()); + assertNotNull(createdChange.getStartTime()); + assertEquals("1", createdChange.getId()); + waitForChangeToComplete(rpc, ZONE1.getName(), "1"); // necessary for the following to return 409 + try { + rpc.applyChangeRequest(ZONE1.getName(), CHANGE1, EMPTY_RPC_OPTIONS); + fail(); + } catch (DnsException ex) { + assertEquals(409, ex.code()); + assertTrue(ex.getMessage().contains("already exists")); + } + assertNotNull(rpc.getChangeRequest(ZONE1.getName(), "1", EMPTY_RPC_OPTIONS)); + assertNull(rpc.getChangeRequest(ZONE1.getName(), "2", EMPTY_RPC_OPTIONS)); + // delete + rpc.applyChangeRequest(ZONE1.getName(), CHANGE2, EMPTY_RPC_OPTIONS); + assertNotNull(rpc.getChangeRequest(ZONE1.getName(), "1", EMPTY_RPC_OPTIONS)); + assertNotNull(rpc.getChangeRequest(ZONE1.getName(), "2", EMPTY_RPC_OPTIONS)); + waitForChangeToComplete(rpc, ZONE1.getName(), "2"); + rpc.applyChangeRequest(ZONE1.getName(), CHANGE_KEEP, EMPTY_RPC_OPTIONS); + waitForChangeToComplete(rpc, ZONE1.getName(), "3"); + Iterable results = + rpc.listRecordSets(ZONE1.getName(), EMPTY_RPC_OPTIONS).results(); + List defaults = ImmutableList.of("SOA", "NS"); + boolean rrsetKeep = false; + boolean rrset1 = false; + for (ResourceRecordSet dnsRecord : results) { + if (dnsRecord.getName().equals(RRSET_KEEP.getName()) + && dnsRecord.getType().equals(RRSET_KEEP.getType())) { + rrsetKeep = true; + } else if (dnsRecord.getName().equals(RRSET1.getName()) + && dnsRecord.getType().equals(RRSET1.getType())) { + rrset1 = true; + } else if (!defaults.contains(dnsRecord.getType())) { + fail(String.format("Record with type %s should not exist", dnsRecord.getType())); + } + } + assertTrue(rrset1); + assertTrue(rrsetKeep); + } + + @Test + public void testGetProject() { + // the projects are automatically created when getProject is called + assertNotNull(LOCAL_DNS_HELPER.getProject(PROJECT_ID1, null)); + assertNotNull(LOCAL_DNS_HELPER.getProject(PROJECT_ID2, null)); + Project project = RPC.getProject(EMPTY_RPC_OPTIONS); + assertNotNull(project.getQuota()); + assertEquals(REAL_PROJECT_ID, project.getId()); + // fields options + Map options = new HashMap<>(); + options.put(DnsRpc.Option.FIELDS, "number"); + project = RPC.getProject(options); + assertNull(project.getId()); + assertNotNull(project.getNumber()); + assertNull(project.getQuota()); + options.put(DnsRpc.Option.FIELDS, "id"); + project = RPC.getProject(options); + assertNotNull(project.getId()); + assertNull(project.getNumber()); + assertNull(project.getQuota()); + options.put(DnsRpc.Option.FIELDS, "quota"); + project = RPC.getProject(options); + assertNull(project.getId()); + assertNull(project.getNumber()); + assertNotNull(project.getQuota()); + } + + @Test + public void testCreateChange() { + // non-existent zone + try { + RPC.applyChangeRequest(ZONE_NAME1, CHANGE1, EMPTY_RPC_OPTIONS); + fail("Zone was not created yet."); + } catch (DnsException ex) { + assertEquals(404, ex.code()); + } + // existent zone + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + assertNull(RPC.getChangeRequest(ZONE_NAME1, "1", EMPTY_RPC_OPTIONS)); + Change created = RPC.applyChangeRequest(ZONE1.getName(), CHANGE1, EMPTY_RPC_OPTIONS); + assertEquals(created, RPC.getChangeRequest(ZONE_NAME1, "1", EMPTY_RPC_OPTIONS)); + // field options + RPC.applyChangeRequest(ZONE1.getName(), CHANGE_KEEP, EMPTY_RPC_OPTIONS); + Map options = new HashMap<>(); + options.put(DnsRpc.Option.FIELDS, "additions"); + Change complex = RPC.applyChangeRequest(ZONE1.getName(), CHANGE_COMPLEX, options); + assertNotNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + options.put(DnsRpc.Option.FIELDS, "deletions"); + complex = RPC.applyChangeRequest(ZONE1.getName(), CHANGE_COMPLEX, options); + assertNull(complex.getAdditions()); + assertNotNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + options.put(DnsRpc.Option.FIELDS, "id"); + complex = RPC.applyChangeRequest(ZONE1.getName(), CHANGE_COMPLEX, options); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNotNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + options.put(DnsRpc.Option.FIELDS, "startTime"); + complex = RPC.applyChangeRequest(ZONE1.getName(), CHANGE_COMPLEX, options); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNotNull(complex.getStartTime()); + assertNull(complex.getStatus()); + options.put(DnsRpc.Option.FIELDS, "status"); + complex = RPC.applyChangeRequest(ZONE1.getName(), CHANGE_COMPLEX, options); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNotNull(complex.getStatus()); + } + + @Test + public void testGetChange() { + // existent + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + Change created = RPC.applyChangeRequest(ZONE1.getName(), CHANGE1, EMPTY_RPC_OPTIONS); + Change retrieved = RPC.getChangeRequest(ZONE1.getName(), "1", EMPTY_RPC_OPTIONS); + assertEquals(created, retrieved); + // non-existent + assertNull(RPC.getChangeRequest(ZONE1.getName(), "2", EMPTY_RPC_OPTIONS)); + // non-existent zone + try { + RPC.getChangeRequest(ZONE_NAME2, "1", EMPTY_RPC_OPTIONS); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(404, ex.code()); + assertTrue(ex.getMessage().contains("managedZone")); + } + // field options + RPC.applyChangeRequest(ZONE1.getName(), CHANGE_KEEP, EMPTY_RPC_OPTIONS); + Change change = RPC.applyChangeRequest(ZONE1.getName(), CHANGE_COMPLEX, EMPTY_RPC_OPTIONS); + Map options = new HashMap<>(); + options.put(DnsRpc.Option.FIELDS, "additions"); + Change complex = RPC.getChangeRequest(ZONE1.getName(), change.getId(), options); + assertNotNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + options.put(DnsRpc.Option.FIELDS, "deletions"); + complex = RPC.getChangeRequest(ZONE1.getName(), change.getId(), options); + assertNull(complex.getAdditions()); + assertNotNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + options.put(DnsRpc.Option.FIELDS, "id"); + complex = RPC.getChangeRequest(ZONE1.getName(), change.getId(), options); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNotNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + options.put(DnsRpc.Option.FIELDS, "startTime"); + complex = RPC.getChangeRequest(ZONE1.getName(), change.getId(), options); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNotNull(complex.getStartTime()); + assertNull(complex.getStatus()); + options.put(DnsRpc.Option.FIELDS, "status"); + complex = RPC.getChangeRequest(ZONE1.getName(), change.getId(), options); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNotNull(complex.getStatus()); + } + + @Test + public void testListZones() { + Iterable results = RPC.listZones(EMPTY_RPC_OPTIONS).results(); + ImmutableList zones = ImmutableList.copyOf(results); + assertEquals(0, zones.size()); + // some zones exists + ManagedZone created = RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + results = RPC.listZones(EMPTY_RPC_OPTIONS).results(); + zones = ImmutableList.copyOf(results); + assertEquals(created, zones.get(0)); + assertEquals(1, zones.size()); + created = RPC.create(ZONE2, EMPTY_RPC_OPTIONS); + results = RPC.listZones(EMPTY_RPC_OPTIONS).results(); + zones = ImmutableList.copyOf(results); + assertEquals(2, zones.size()); + assertTrue(zones.contains(created)); + // error in options + Map options = new HashMap<>(); + options.put(DnsRpc.Option.PAGE_SIZE, 0); + try { + RPC.listZones(options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.maxResults")); + } + options = new HashMap<>(); + options.put(DnsRpc.Option.PAGE_SIZE, -1); + try { + RPC.listZones(options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.maxResults")); + } + // ok size + options = new HashMap<>(); + options.put(DnsRpc.Option.PAGE_SIZE, 335); + results = RPC.listZones(options).results(); + zones = ImmutableList.copyOf(results); + assertEquals(2, zones.size()); + // dns name problems + options = new HashMap<>(); + options.put(DnsRpc.Option.DNS_NAME, "aaa"); + try { + RPC.listZones(options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.dnsName")); + } + // ok name + options = new HashMap<>(); + options.put(DnsRpc.Option.DNS_NAME, "aaaa."); + results = RPC.listZones(options).results(); + zones = ImmutableList.copyOf(results); + assertEquals(0, zones.size()); + // field options + options = new HashMap<>(); + options.put(DnsRpc.Option.FIELDS, "managedZones(id)"); + ManagedZone zone = RPC.listZones(options).results().iterator().next(); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNotNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "managedZones(creationTime)"); + zone = RPC.listZones(options).results().iterator().next(); + assertNotNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "managedZones(dnsName)"); + zone = RPC.listZones(options).results().iterator().next(); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNotNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "managedZones(description)"); + zone = RPC.listZones(options).results().iterator().next(); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNotNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "managedZones(nameServers)"); + zone = RPC.listZones(options).results().iterator().next(); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNotNull(zone.getNameServers()); + assertNull(zone.getNameServerSet()); + assertNull(zone.getId()); + options.put(DnsRpc.Option.FIELDS, "managedZones(nameServerSet)"); + DnsRpc.ListResult listResult = RPC.listZones(options); + zone = listResult.results().iterator().next(); + assertNull(listResult.pageToken()); + assertNull(zone.getCreationTime()); + assertNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNotNull(zone.getNameServerSet()); + assertNull(zone.getId()); + // several combined + options.put(DnsRpc.Option.FIELDS, + "managedZones(nameServerSet,description,id,name),nextPageToken"); + options.put(DnsRpc.Option.PAGE_SIZE, 1); + listResult = RPC.listZones(options); + zone = listResult.results().iterator().next(); + assertNull(zone.getCreationTime()); + assertNotNull(zone.getName()); + assertNull(zone.getDnsName()); + assertNotNull(zone.getDescription()); + assertNull(zone.getNameServers()); + assertNotNull(zone.getNameServerSet()); + assertNotNull(zone.getId()); + assertEquals(zone.getName(), listResult.pageToken()); + } + + @Test + public void testListDnsRecords() { + // no zone exists + try { + RPC.listRecordSets(ZONE_NAME1, EMPTY_RPC_OPTIONS); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(404, ex.code()); + assertTrue(ex.getMessage().contains("managedZone")); + } + // zone exists but has no records + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + Iterable results = + RPC.listRecordSets(ZONE_NAME1, EMPTY_RPC_OPTIONS).results(); + ImmutableList records = ImmutableList.copyOf(results); + assertEquals(2, records.size()); // contains default NS and SOA + // zone has records + RPC.applyChangeRequest(ZONE_NAME1, CHANGE_KEEP, EMPTY_RPC_OPTIONS); + results = RPC.listRecordSets(ZONE_NAME1, EMPTY_RPC_OPTIONS).results(); + records = ImmutableList.copyOf(results); + assertEquals(3, records.size()); + // error in options + Map options = new HashMap<>(); + options.put(DnsRpc.Option.PAGE_SIZE, 0); + try { + RPC.listRecordSets(ZONE1.getName(), options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.maxResults")); + } + options.put(DnsRpc.Option.PAGE_SIZE, -1); + try { + RPC.listRecordSets(ZONE1.getName(), options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.maxResults")); + } + options.put(DnsRpc.Option.PAGE_SIZE, 15); + results = RPC.listRecordSets(ZONE1.getName(), options).results(); + records = ImmutableList.copyOf(results); + assertEquals(3, records.size()); + // dnsName filter + options = new HashMap<>(); + options.put(DnsRpc.Option.NAME, "aaa"); + try { + RPC.listRecordSets(ZONE1.getName(), options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.name")); + } + options.put(DnsRpc.Option.NAME, "aaa."); + results = RPC.listRecordSets(ZONE1.getName(), options).results(); + records = ImmutableList.copyOf(results); + assertEquals(0, records.size()); + options.put(DnsRpc.Option.NAME, null); + options.put(DnsRpc.Option.DNS_TYPE, "A"); + try { + RPC.listRecordSets(ZONE1.getName(), options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.name")); + } + options.put(DnsRpc.Option.NAME, "aaa."); + options.put(DnsRpc.Option.DNS_TYPE, "a"); + try { + RPC.listRecordSets(ZONE1.getName(), options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.type")); + } + options.put(DnsRpc.Option.NAME, DNS_NAME); + options.put(DnsRpc.Option.DNS_TYPE, "SOA"); + results = RPC.listRecordSets(ZONE1.getName(), options).results(); + records = ImmutableList.copyOf(results); + assertEquals(1, records.size()); + // field options + options = new HashMap<>(); + options.put(DnsRpc.Option.FIELDS, "rrsets(name)"); + DnsRpc.ListResult listResult = + RPC.listRecordSets(ZONE1.getName(), options); + records = ImmutableList.copyOf(listResult.results()); + ResourceRecordSet record = records.get(0); + assertNotNull(record.getName()); + assertNull(record.getRrdatas()); + assertNull(record.getType()); + assertNull(record.getTtl()); + assertNull(listResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "rrsets(rrdatas)"); + listResult = RPC.listRecordSets(ZONE1.getName(), options); + records = ImmutableList.copyOf(listResult.results()); + record = records.get(0); + assertNull(record.getName()); + assertNotNull(record.getRrdatas()); + assertNull(record.getType()); + assertNull(record.getTtl()); + assertNull(listResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "rrsets(ttl)"); + listResult = RPC.listRecordSets(ZONE1.getName(), options); + records = ImmutableList.copyOf(listResult.results()); + record = records.get(0); + assertNull(record.getName()); + assertNull(record.getRrdatas()); + assertNull(record.getType()); + assertNotNull(record.getTtl()); + assertNull(listResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "rrsets(type)"); + listResult = RPC.listRecordSets(ZONE1.getName(), options); + records = ImmutableList.copyOf(listResult.results()); + record = records.get(0); + assertNull(record.getName()); + assertNull(record.getRrdatas()); + assertNotNull(record.getType()); + assertNull(record.getTtl()); + assertNull(listResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "nextPageToken"); + listResult = RPC.listRecordSets(ZONE1.getName(), options); + records = ImmutableList.copyOf(listResult.results()); + record = records.get(0); + assertNull(record.getName()); + assertNull(record.getRrdatas()); + assertNull(record.getType()); + assertNull(record.getTtl()); + assertNull(listResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "nextPageToken,rrsets(name,rrdatas)"); + options.put(DnsRpc.Option.PAGE_SIZE, 1); + listResult = RPC.listRecordSets(ZONE1.getName(), options); + records = ImmutableList.copyOf(listResult.results()); + assertEquals(1, records.size()); + record = records.get(0); + assertNotNull(record.getName()); + assertNotNull(record.getRrdatas()); + assertNull(record.getType()); + assertNull(record.getTtl()); + assertNotNull(listResult.pageToken()); + } + + @Test + public void testListChanges() { + // no such zone exists + try { + RPC.listChangeRequests(ZONE_NAME1, EMPTY_RPC_OPTIONS); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(404, ex.code()); + assertTrue(ex.getMessage().contains("managedZone")); + } + // zone exists but has no changes + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + Iterable results = RPC.listChangeRequests(ZONE1.getName(), EMPTY_RPC_OPTIONS).results(); + ImmutableList changes = ImmutableList.copyOf(results); + assertEquals(1, changes.size()); + // zone has changes + RPC.applyChangeRequest(ZONE1.getName(), CHANGE1, EMPTY_RPC_OPTIONS); + RPC.applyChangeRequest(ZONE1.getName(), CHANGE2, EMPTY_RPC_OPTIONS); + RPC.applyChangeRequest(ZONE1.getName(), CHANGE_KEEP, EMPTY_RPC_OPTIONS); + results = RPC.listChangeRequests(ZONE1.getName(), EMPTY_RPC_OPTIONS).results(); + changes = ImmutableList.copyOf(results); + assertEquals(4, changes.size()); + // error in options + Map options = new HashMap<>(); + options.put(DnsRpc.Option.PAGE_SIZE, 0); + try { + RPC.listChangeRequests(ZONE1.getName(), options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.maxResults")); + } + options.put(DnsRpc.Option.PAGE_SIZE, -1); + try { + RPC.listChangeRequests(ZONE1.getName(), options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.maxResults")); + } + options.put(DnsRpc.Option.PAGE_SIZE, 15); + results = RPC.listChangeRequests(ZONE1.getName(), options).results(); + changes = ImmutableList.copyOf(results); + assertEquals(4, changes.size()); + options = new HashMap<>(); + options.put(DnsRpc.Option.SORTING_ORDER, "descending"); + results = RPC.listChangeRequests(ZONE1.getName(), options).results(); + ImmutableList descending = ImmutableList.copyOf(results); + results = RPC.listChangeRequests(ZONE1.getName(), EMPTY_RPC_OPTIONS).results(); + ImmutableList ascending = ImmutableList.copyOf(results); + int size = 4; + assertEquals(size, descending.size()); + for (int i = 0; i < size; i++) { + assertEquals(descending.get(i), ascending.get(size - i - 1)); + } + options.put(DnsRpc.Option.SORTING_ORDER, "something else"); + try { + RPC.listChangeRequests(ZONE1.getName(), options); + fail(); + } catch (DnsException ex) { + // expected + assertEquals(400, ex.code()); + assertTrue(ex.getMessage().contains("parameters.sortOrder")); + } + // field options + RPC.applyChangeRequest(ZONE1.getName(), CHANGE_COMPLEX, EMPTY_RPC_OPTIONS); + options = new HashMap<>(); + options.put(DnsRpc.Option.SORTING_ORDER, "descending"); + options.put(DnsRpc.Option.FIELDS, "changes(additions)"); + DnsRpc.ListResult changeListResult = RPC.listChangeRequests(ZONE1.getName(), options); + changes = ImmutableList.copyOf(changeListResult.results()); + Change complex = changes.get(0); + assertNotNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + assertNull(changeListResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "changes(deletions)"); + changeListResult = RPC.listChangeRequests(ZONE1.getName(), options); + changes = ImmutableList.copyOf(changeListResult.results()); + complex = changes.get(0); + assertNull(complex.getAdditions()); + assertNotNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + assertNull(changeListResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "changes(id)"); + changeListResult = RPC.listChangeRequests(ZONE1.getName(), options); + changes = ImmutableList.copyOf(changeListResult.results()); + complex = changes.get(0); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNotNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + assertNull(changeListResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "changes(startTime)"); + changeListResult = RPC.listChangeRequests(ZONE1.getName(), options); + changes = ImmutableList.copyOf(changeListResult.results()); + complex = changes.get(0); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNotNull(complex.getStartTime()); + assertNull(complex.getStatus()); + assertNull(changeListResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "changes(status)"); + changeListResult = RPC.listChangeRequests(ZONE1.getName(), options); + changes = ImmutableList.copyOf(changeListResult.results()); + complex = changes.get(0); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNotNull(complex.getStatus()); + assertNull(changeListResult.pageToken()); + options.put(DnsRpc.Option.FIELDS, "nextPageToken"); + options.put(DnsRpc.Option.PAGE_SIZE, 1); + changeListResult = RPC.listChangeRequests(ZONE1.getName(), options); + changes = ImmutableList.copyOf(changeListResult.results()); + complex = changes.get(0); + assertNull(complex.getAdditions()); + assertNull(complex.getDeletions()); + assertNull(complex.getId()); + assertNull(complex.getStartTime()); + assertNull(complex.getStatus()); + assertNotNull(changeListResult.pageToken()); + } + + @Test + public void testDnsRecordPaging() { + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + List complete = ImmutableList.copyOf( + RPC.listRecordSets(ZONE1.getName(), EMPTY_RPC_OPTIONS).results()); + Map options = new HashMap<>(); + options.put(DnsRpc.Option.PAGE_SIZE, 1); + DnsRpc.ListResult listResult = RPC.listRecordSets(ZONE1.getName(), options); + ImmutableList records = ImmutableList.copyOf(listResult.results()); + assertEquals(1, records.size()); + assertEquals(complete.get(0), records.get(0)); + options.put(DnsRpc.Option.PAGE_TOKEN, listResult.pageToken()); + listResult = RPC.listRecordSets(ZONE1.getName(), options); + records = ImmutableList.copyOf(listResult.results()); + assertEquals(1, records.size()); + assertEquals(complete.get(1), records.get(0)); + } + + @Test + public void testZonePaging() { + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + RPC.create(ZONE2, EMPTY_RPC_OPTIONS); + ImmutableList complete = ImmutableList.copyOf( + RPC.listZones(EMPTY_RPC_OPTIONS).results()); + Map options = new HashMap<>(); + options.put(DnsRpc.Option.PAGE_SIZE, 1); + DnsRpc.ListResult listResult = RPC.listZones(options); + ImmutableList page1 = ImmutableList.copyOf(listResult.results()); + assertEquals(1, page1.size()); + assertEquals(complete.get(0), page1.get(0)); + assertEquals(page1.get(0).getName(), listResult.pageToken()); + options.put(DnsRpc.Option.PAGE_TOKEN, listResult.pageToken()); + listResult = RPC.listZones(options); + ImmutableList page2 = ImmutableList.copyOf(listResult.results()); + assertEquals(1, page2.size()); + assertEquals(complete.get(1), page2.get(0)); + assertNull(listResult.pageToken()); + } + + @Test + public void testChangePaging() { + RPC.create(ZONE1, EMPTY_RPC_OPTIONS); + RPC.applyChangeRequest(ZONE1.getName(), CHANGE1, EMPTY_RPC_OPTIONS); + RPC.applyChangeRequest(ZONE1.getName(), CHANGE2, EMPTY_RPC_OPTIONS); + RPC.applyChangeRequest(ZONE1.getName(), CHANGE_KEEP, EMPTY_RPC_OPTIONS); + ImmutableList complete = + ImmutableList.copyOf(RPC.listChangeRequests(ZONE1.getName(), EMPTY_RPC_OPTIONS).results()); + Map options = new HashMap<>(); + options.put(DnsRpc.Option.PAGE_SIZE, 1); + DnsRpc.ListResult changeListResult = RPC.listChangeRequests(ZONE1.getName(), options); + List changes = ImmutableList.copyOf(changeListResult.results()); + assertEquals(1, changes.size()); + assertEquals(complete.get(0), changes.get(0)); + assertEquals(complete.get(0).getId(), changeListResult.pageToken()); + options.put(DnsRpc.Option.PAGE_TOKEN, changeListResult.pageToken()); + changeListResult = RPC.listChangeRequests(ZONE1.getName(), options); + changes = ImmutableList.copyOf(changeListResult.results()); + assertEquals(1, changes.size()); + assertEquals(complete.get(1), changes.get(0)); + assertEquals(complete.get(1).getId(), changeListResult.pageToken()); + } + + @Test + public void testToListResponse() { + LocalDnsHelper.Response response = LocalDnsHelper.toListResponse( + Lists.newArrayList("some", "multiple", "words"), "contextA", "IncludeThisPageToken", true); + assertTrue(response.body().contains("IncludeThisPageToken")); + assertTrue(response.body().contains("contextA")); + response = LocalDnsHelper.toListResponse( + Lists.newArrayList("some", "multiple", "words"), "contextB", "IncludeThisPageToken", false); + assertFalse(response.body().contains("IncludeThisPageToken")); + assertTrue(response.body().contains("contextB")); + response = LocalDnsHelper.toListResponse( + Lists.newArrayList("some", "multiple", "words"), "contextC", null, true); + assertFalse(response.body().contains("pageToken")); + assertTrue(response.body().contains("contextC")); + } + + @Test + public void testCreateZoneValidation() { + ManagedZone minimalZone = copyZone(ZONE1); + // no name + ManagedZone copy = copyZone(minimalZone); + copy.setName(null); + LocalDnsHelper.Response response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("entity.managedZone.name")); + // no description + copy = copyZone(minimalZone); + copy.setDescription(null); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("entity.managedZone.description")); + // no dns name + copy = copyZone(minimalZone); + copy.setDnsName(null); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("entity.managedZone.dnsName")); + // zone name does not start with a letter + copy = copyZone(minimalZone); + copy.setName("1aaaaaa"); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("entity.managedZone.name")); + assertTrue(response.body().contains("Invalid")); + // zone name is too long + copy = copyZone(minimalZone); + copy.setName("123456aaaa123456aaaa123456aaaa123456aaaa123456aaaa123456aaaa123456aaaa123456aa"); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("entity.managedZone.name")); + assertTrue(response.body().contains("Invalid")); + // zone name contains invalid characters + copy = copyZone(minimalZone); + copy.setName("x1234AA6aa"); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("entity.managedZone.name")); + assertTrue(response.body().contains("Invalid")); + // zone name contains invalid characters + copy = copyZone(minimalZone); + copy.setName("x a"); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("entity.managedZone.name")); + assertTrue(response.body().contains("Invalid")); + // dns name does not end with period + copy = copyZone(minimalZone); + copy.setDnsName("aaaaaa.com"); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("entity.managedZone.dnsName")); + assertTrue(response.body().contains("Invalid")); + // dns name is reserved + copy = copyZone(minimalZone); + copy.setDnsName("com."); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(400, response.code()); + assertTrue(response.body().contains("not available to be created.")); + // empty description should pass + copy = copyZone(minimalZone); + copy.setDescription(""); + response = LOCAL_DNS_HELPER.createZone(PROJECT_ID1, copy); + assertEquals(200, response.code()); + } + + @Test + public void testCheckListOptions() { + // listing zones + optionsMap.put("maxResults", "-1"); + LocalDnsHelper.Response response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("parameters.maxResults")); + optionsMap.put("maxResults", "0"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("parameters.maxResults")); + optionsMap.put("maxResults", "aaaa"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("integer")); + optionsMap.put("maxResults", "15"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertNull(response); + optionsMap.put("dnsName", "aaa"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("parameters.dnsName")); + optionsMap.put("dnsName", "aaa."); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertNull(response); + // listing dns records + optionsMap.put("name", "aaa"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("parameters.name")); + optionsMap.put("name", "aaa."); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertNull(response); + optionsMap.put("name", null); + optionsMap.put("type", "A"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("parameters.name")); + optionsMap.put("name", "aaa."); + optionsMap.put("type", "a"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("parameters.type")); + optionsMap.put("name", "aaaa."); + optionsMap.put("type", "A"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertNull(response); + // listing changes + optionsMap.put("sortBy", "changeSequence"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertNull(response); + optionsMap.put("sortBy", "something else"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("Allowed values: [changesequence]")); + optionsMap.put("sortBy", "ChAnGeSeQuEnCe"); // is not case sensitive + response = LocalDnsHelper.checkListOptions(optionsMap); + assertNull(response); + optionsMap.put("sortOrder", "ascending"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertNull(response); + optionsMap.put("sortOrder", "descending"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertNull(response); + optionsMap.put("sortOrder", "somethingelse"); + response = LocalDnsHelper.checkListOptions(optionsMap); + assertEquals(400, response.code()); + assertTrue(response.body().contains("parameters.sortOrder")); + } + + @Test + public void testCheckRrset() { + ResourceRecordSet valid = new ResourceRecordSet(); + valid.setName(ZONE1.getDnsName()); + valid.setType("A"); + valid.setRrdatas(ImmutableList.of("0.255.1.5")); + valid.setTtl(500); + Change validChange = new Change(); + validChange.setAdditions(ImmutableList.of(valid)); + LOCAL_DNS_HELPER.createZone(PROJECT_ID1, ZONE1); + LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, validChange); + // delete with field mismatch + LocalDnsHelper.ZoneContainer zone = LOCAL_DNS_HELPER.findZone(PROJECT_ID1, ZONE_NAME1); + valid.setTtl(valid.getTtl() + 20); + LocalDnsHelper.Response response = LocalDnsHelper.checkRrset(valid, zone, 0, "deletions"); + assertEquals(412, response.code()); + assertTrue(response.body().contains("entity.change.deletions[0]")); + } + + @Test + public void testCheckRrdata() { + // A + assertTrue(LocalDnsHelper.checkRrData("255.255.255.255", "A")); + assertTrue(LocalDnsHelper.checkRrData("13.15.145.165", "A")); + assertTrue(LocalDnsHelper.checkRrData("0.0.0.0", "A")); + assertFalse(LocalDnsHelper.checkRrData("255.255.255.256", "A")); + assertFalse(LocalDnsHelper.checkRrData("-1.255.255.255", "A")); + assertFalse(LocalDnsHelper.checkRrData(".255.255.254", "A")); + assertFalse(LocalDnsHelper.checkRrData("111.255.255.", "A")); + assertFalse(LocalDnsHelper.checkRrData("111.255..22", "A")); + assertFalse(LocalDnsHelper.checkRrData("111.255.aa.22", "A")); + assertFalse(LocalDnsHelper.checkRrData("", "A")); + assertFalse(LocalDnsHelper.checkRrData("...", "A")); + assertFalse(LocalDnsHelper.checkRrData("111.255.12", "A")); + assertFalse(LocalDnsHelper.checkRrData("111.255.12.11.11", "A")); + // AAAA + assertTrue(LocalDnsHelper.checkRrData("1F:fa:09fd::343:aaaa:AAAA:0", "AAAA")); + assertTrue(LocalDnsHelper.checkRrData("0000:FFFF:09fd::343:aaaa:AAAA:0", "AAAA")); + assertFalse(LocalDnsHelper.checkRrData("-2:::::::", "AAAA")); + assertTrue(LocalDnsHelper.checkRrData("0::0", "AAAA")); + assertFalse(LocalDnsHelper.checkRrData("::1FFFF:::::", "AAAA")); + assertFalse(LocalDnsHelper.checkRrData("::aqaa:::::", "AAAA")); + assertFalse(LocalDnsHelper.checkRrData("::::::::", "AAAA")); // too long + assertFalse(LocalDnsHelper.checkRrData("::::::", "AAAA")); // too short + } + + @Test + public void testCheckChange() { + ResourceRecordSet validA = new ResourceRecordSet(); + validA.setName(ZONE1.getDnsName()); + validA.setType("A"); + validA.setRrdatas(ImmutableList.of("0.255.1.5")); + ResourceRecordSet invalidA = new ResourceRecordSet(); + invalidA.setName(ZONE1.getDnsName()); + invalidA.setType("A"); + invalidA.setRrdatas(ImmutableList.of("0.-255.1.5")); + Change validChange = new Change(); + validChange.setAdditions(ImmutableList.of(validA)); + Change invalidChange = new Change(); + invalidChange.setAdditions(ImmutableList.of(invalidA)); + LocalDnsHelper.ZoneContainer zoneContainer = new LocalDnsHelper.ZoneContainer(ZONE1); + LocalDnsHelper.Response response = LocalDnsHelper.checkChange(validChange, zoneContainer); + assertNull(response); + response = LocalDnsHelper.checkChange(invalidChange, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains("additions[0].rrdata[0]")); + // only empty additions/deletions + Change empty = new Change(); + empty.setAdditions(ImmutableList.of()); + empty.setDeletions(ImmutableList.of()); + response = LocalDnsHelper.checkChange(empty, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains( + "The 'entity.change' parameter is required but was missing.")); + // null additions/deletions + empty = new Change(); + response = LocalDnsHelper.checkChange(empty, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains( + "The 'entity.change' parameter is required but was missing.")); + // non-matching name + validA.setName(ZONE1.getDnsName() + ".aaa."); + response = LocalDnsHelper.checkChange(validChange, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains("additions[0].name")); + // wrong type + validA.setName(ZONE1.getDnsName()); // revert + validA.setType("ABCD"); + response = LocalDnsHelper.checkChange(validChange, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains("additions[0].type")); + // wrong ttl + validA.setType("A"); // revert + validA.setTtl(-1); + response = LocalDnsHelper.checkChange(validChange, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains("additions[0].ttl")); + validA.setTtl(null); + // null name + validA.setName(null); + response = LocalDnsHelper.checkChange(validChange, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains("additions[0].name")); + validA.setName(ZONE1.getDnsName()); + // null type + validA.setType(null); + response = LocalDnsHelper.checkChange(validChange, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains("additions[0].type")); + validA.setType("A"); + // null rrdata + final List temp = validA.getRrdatas(); // preserve + validA.setRrdatas(null); + response = LocalDnsHelper.checkChange(validChange, zoneContainer); + assertEquals(400, response.code()); + assertTrue(response.body().contains("additions[0].rrdata")); + validA.setRrdatas(temp); + // delete non-existent + ResourceRecordSet nonExistent = new ResourceRecordSet(); + nonExistent.setName(ZONE1.getDnsName()); + nonExistent.setType("AAAA"); + nonExistent.setRrdatas(ImmutableList.of("0:0:0:0:5::6")); + Change delete = new Change(); + delete.setDeletions(ImmutableList.of(nonExistent)); + response = LocalDnsHelper.checkChange(delete, zoneContainer); + assertEquals(404, response.code()); + assertTrue(response.body().contains("deletions[0]")); + } + + @Test + public void testCheckAdditionsDeletions() { + ResourceRecordSet validA = new ResourceRecordSet(); + validA.setName(ZONE1.getDnsName()); + validA.setType("A"); + validA.setRrdatas(ImmutableList.of("0.255.1.5")); + Change validChange = new Change(); + validChange.setAdditions(ImmutableList.of(validA)); + LOCAL_DNS_HELPER.createZone(PROJECT_ID1, ZONE1); + LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, validChange); + LocalDnsHelper.ZoneContainer container = LOCAL_DNS_HELPER.findZone(PROJECT_ID1, ZONE_NAME1); + LocalDnsHelper.Response response = + LocalDnsHelper.checkAdditionsDeletions(ImmutableList.of(validA), null, container); + assertEquals(409, response.code()); + assertTrue(response.body().contains("already exists")); + } + + @Test + public void testCreateChangeContentValidation() { + ResourceRecordSet validA = new ResourceRecordSet(); + validA.setName(ZONE1.getDnsName()); + validA.setType("A"); + validA.setRrdatas(ImmutableList.of("0.255.1.5")); + Change validChange = new Change(); + validChange.setAdditions(ImmutableList.of(validA)); + LOCAL_DNS_HELPER.createZone(PROJECT_ID1, ZONE1); + LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, validChange); + LocalDnsHelper.Response response = + LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, validChange); + assertEquals(409, response.code()); + assertTrue(response.body().contains("already exists")); + // delete with field mismatch + Change delete = new Change(); + validA.setTtl(20); // mismatch + delete.setDeletions(ImmutableList.of(validA)); + response = LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, delete); + assertEquals(412, response.code()); + assertTrue(response.body().contains("entity.change.deletions[0]")); + // delete and add SOA + Change addition = new Change(); + ImmutableCollection dnsRecords = + LOCAL_DNS_HELPER.findZone(PROJECT_ID1, ZONE_NAME1).dnsRecords().get().values(); + LinkedList deletions = new LinkedList<>(); + LinkedList additions = new LinkedList<>(); + for (ResourceRecordSet rrset : dnsRecords) { + if (rrset.getType().equals("SOA")) { + deletions.add(rrset); + ResourceRecordSet copy = copyRrset(rrset); + copy.setName("x." + copy.getName()); + additions.add(copy); + break; + } + } + delete.setDeletions(deletions); + addition.setAdditions(additions); + response = LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, delete); + assertEquals(400, response.code()); + assertTrue(response.body().contains( + "zone must contain exactly one resource record set of type 'SOA' at the apex")); + assertTrue(response.body().contains("deletions[0]")); + response = LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, addition); + assertEquals(400, response.code()); + assertTrue(response.body().contains( + "zone must contain exactly one resource record set of type 'SOA' at the apex")); + assertTrue(response.body().contains("additions[0]")); + // delete NS + deletions = new LinkedList<>(); + additions = new LinkedList<>(); + for (ResourceRecordSet rrset : dnsRecords) { + if (rrset.getType().equals("NS")) { + deletions.add(rrset); + ResourceRecordSet copy = copyRrset(rrset); + copy.setName("x." + copy.getName()); + additions.add(copy); + break; + } + } + delete.setDeletions(deletions); + addition.setAdditions(additions); + response = LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, delete); + assertEquals(400, response.code()); + assertTrue(response.body().contains( + "zone must contain exactly one resource record set of type 'NS' at the apex")); + response = LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, addition); + assertEquals(400, response.code()); + assertTrue(response.body().contains( + "zone must contain exactly one resource record set of type 'NS' at the apex")); + assertTrue(response.body().contains("additions[0]")); + // change (delete + add) + addition.setDeletions(deletions); + response = LOCAL_DNS_HELPER.createChange(PROJECT_ID1, ZONE_NAME1, addition); + assertEquals(200, response.code()); + } + + @Test + public void testMatchesCriteria() { + assertTrue(LocalDnsHelper.matchesCriteria(RRSET1, RRSET1.getName(), RRSET1.getType())); + assertFalse(LocalDnsHelper.matchesCriteria(RRSET1, RRSET1.getName(), "anothertype")); + assertTrue(LocalDnsHelper.matchesCriteria(RRSET1, null, RRSET1.getType())); + assertTrue(LocalDnsHelper.matchesCriteria(RRSET1, RRSET1.getName(), null)); + assertFalse(LocalDnsHelper.matchesCriteria(RRSET1, "anothername", RRSET1.getType())); + } + + @Test + public void testGetUniqueId() { + assertNotNull(LocalDnsHelper.getUniqueId(new HashSet())); + } + + @Test + public void testRandomNameServers() { + assertEquals(4, LocalDnsHelper.randomNameservers().size()); + } + + private static ManagedZone copyZone(ManagedZone original) { + ManagedZone copy = new ManagedZone(); + copy.setDescription(original.getDescription()); + copy.setName(original.getName()); + copy.setCreationTime(original.getCreationTime()); + copy.setId(original.getId()); + copy.setNameServerSet(original.getNameServerSet()); + copy.setDnsName(original.getDnsName()); + if (original.getNameServers() != null) { + copy.setNameServers(ImmutableList.copyOf(original.getNameServers())); + } + return copy; + } + + private static ResourceRecordSet copyRrset(ResourceRecordSet set) { + ResourceRecordSet copy = new ResourceRecordSet(); + if (set.getRrdatas() != null) { + copy.setRrdatas(ImmutableList.copyOf(set.getRrdatas())); + } + copy.setTtl(set.getTtl()); + copy.setName(set.getName()); + copy.setType(set.getType()); + return copy; + } +} diff --git a/gcloud-java-examples/README.md b/gcloud-java-examples/README.md index 7d24febeac80..8084cee562f0 100644 --- a/gcloud-java-examples/README.md +++ b/gcloud-java-examples/README.md @@ -7,6 +7,7 @@ Examples for gcloud-java (Java idiomatic client for [Google Cloud Platform][clou [![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) [![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-examples.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-examples.svg) [![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) - [Homepage] (https://googlecloudplatform.github.io/gcloud-java/) - [Examples] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/examples/package-summary.html) @@ -18,16 +19,16 @@ If you are using Maven, add this to your pom.xml file com.google.gcloud gcloud-java-examples - 0.1.3 + 0.1.5 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.gcloud:gcloud-java-examples:0.1.3' +compile 'com.google.gcloud:gcloud-java-examples:0.1.5' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.gcloud" % "gcloud-java-examples" % "0.1.3" +libraryDependencies += "com.google.gcloud" % "gcloud-java-examples" % "0.1.5" ``` To run examples from your command line: @@ -62,7 +63,7 @@ To run examples from your command line: ``` * Here's an example run of `DatastoreExample`. - + Be sure to change the placeholder project ID "your-project-id" with your own project ID. Also note that you have to enable the Google Cloud Datastore API on the [Google Developers Console][developers-console] before running the following commands. ``` mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.datastore.DatastoreExample" -Dexec.args="your-project-id my_name add my\ comment" @@ -70,6 +71,22 @@ To run examples from your command line: mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.datastore.DatastoreExample" -Dexec.args="your-project-id my_name delete" ``` + * Here's an example run of `DnsExample`. + + Note that you have to enable the Google Cloud DNS API on the [Google Developers Console][developers-console] before running the following commands. + You will need to replace the domain name `elaborateexample.com` with your own domain name with [verified ownership] (https://www.google.com/webmasters/verification/home). + Also, note that the example creates and deletes record sets of type A only. Operations with other record types are not implemented in the example. + ``` + mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" -Dexec.args="create some-sample-zone elaborateexample.com. description" + mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" -Dexec.args="list" + mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" -Dexec.args="list some-sample-zone records" + mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" -Dexec.args="add-record some-sample-zone www.elaborateexample.com. 12.13.14.15 69" + mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" -Dexec.args="get some-sample-zone" + mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" -Dexec.args="delete-record some-sample-zone www.elaborateexample.com. 12.13.14.15 69" + mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" -Dexec.args="list some-sample-zone changes ascending" + mvn exec:java -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" -Dexec.args="delete some-sample-zone" + ``` + * Here's an example run of `ResourceManagerExample`. Be sure to change the placeholder project ID "your-project-id" with your own globally unique project ID. diff --git a/gcloud-java-examples/pom.xml b/gcloud-java-examples/pom.xml index e5a95f67a9eb..111308658c2e 100644 --- a/gcloud-java-examples/pom.xml +++ b/gcloud-java-examples/pom.xml @@ -10,7 +10,7 @@ com.google.gcloud gcloud-java-pom - 0.1.4-SNAPSHOT + 0.1.6-SNAPSHOT gcloud-java-examples diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/BigQueryExample.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/BigQueryExample.java index c8fbe7289f9c..fe27ee3cf63b 100644 --- a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/BigQueryExample.java +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/BigQueryExample.java @@ -43,7 +43,7 @@ import com.google.gcloud.bigquery.TableInfo; import com.google.gcloud.bigquery.ViewDefinition; import com.google.gcloud.bigquery.WriteChannelConfiguration; -import com.google.gcloud.spi.BigQueryRpc.Tuple; +import com.google.gcloud.bigquery.spi.BigQueryRpc.Tuple; import java.nio.channels.FileChannel; import java.nio.file.Paths; diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/snippets/InsertDataAndQueryTable.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/snippets/InsertDataAndQueryTable.java index f421bc832441..ba2d1291b229 100644 --- a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/snippets/InsertDataAndQueryTable.java +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/bigquery/snippets/InsertDataAndQueryTable.java @@ -84,7 +84,7 @@ public static void main(String... args) throws InterruptedException { // Create a query request QueryRequest queryRequest = QueryRequest.builder("SELECT * FROM my_dataset_id.my_table_id") .maxWaitTime(60000L) - .maxResults(1000L) + .pageSize(1000L) .build(); // Request query to be executed and wait for results QueryResponse queryResponse = bigquery.query(queryRequest); diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/DnsExample.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/DnsExample.java new file mode 100644 index 000000000000..a9e5c5d25377 --- /dev/null +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/DnsExample.java @@ -0,0 +1,525 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.examples.dns; + +import com.google.common.base.Joiner; +import com.google.common.collect.ImmutableList; +import com.google.gcloud.dns.ChangeRequest; +import com.google.gcloud.dns.ChangeRequestInfo; +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.dns.ProjectInfo; +import com.google.gcloud.dns.RecordSet; +import com.google.gcloud.dns.Zone; +import com.google.gcloud.dns.ZoneInfo; + +import java.text.DateFormat; +import java.text.SimpleDateFormat; +import java.util.Arrays; +import java.util.Date; +import java.util.HashMap; +import java.util.Iterator; +import java.util.Map; +import java.util.concurrent.TimeUnit; + +/** + * An example of using Google Cloud DNS. + * + *

This example creates, deletes, gets, and lists zones. It also creates and deletes + * record sets of type A, and lists record sets. + * + *

Steps needed for running the example: + *

    + *
  1. login using gcloud SDK - {@code gcloud auth login}.
  2. + *
  3. compile using maven - {@code mvn compile}
  4. + *
  5. run using maven - {@code mvn exec:java + * -Dexec.mainClass="com.google.gcloud.examples.dns.DnsExample" + * -Dexec.args="[] + * create | + * get | + * delete | + * list [ [changes [descending | ascending] | records]] | + * add-record | + * delete-record [] | + * quota}
  6. + *
+ * + *

The first parameter is an optional {@code project_id}. The project specified in the Google + * Cloud SDK configuration (see {@code gcloud config list}) will be used if the project ID is not + * supplied. The second parameter is a DNS operation (list, delete, create, ...). The remaining + * arguments are specific to the operation. See each action's {@code run} method for the specific + * arguments. + */ +public class DnsExample { + + private static final Map ACTIONS = new HashMap<>(); + private static final DateFormat FORMATTER = new SimpleDateFormat("YYYY-MM-dd HH:mm:ss"); + + private interface DnsAction { + void run(Dns dns, String... args); + + String params(); + + boolean check(String... args); + } + + private static class CreateZoneAction implements DnsAction { + + /** + * Creates a zone with the provided name, DNS name and description (in this order). + */ + @Override + public void run(Dns dns, String... args) { + String zoneName = args[0]; + String dnsName = args[1]; + String description = args[2]; + ZoneInfo zoneInfo = ZoneInfo.of(zoneName, dnsName, description); + Zone zone = dns.create(zoneInfo); + System.out.printf("Successfully created zone with name %s which was assigned ID %s.%n", + zone.name(), zone.id()); + } + + @Override + public String params() { + return " "; + } + + @Override + public boolean check(String... args) { + return args.length == 3; + } + } + + private static class ListZonesAction implements DnsAction { + + /** + * Lists all zones within the project. + */ + @Override + public void run(Dns dns, String... args) { + Iterator zoneIterator = dns.listZones().iterateAll(); + if (zoneIterator.hasNext()) { + System.out.println("The project contains the following zones:"); + while (zoneIterator.hasNext()) { + printZone(zoneIterator.next()); + } + } else { + System.out.println("Project contains no zones."); + } + } + + @Override + public String params() { + return ""; + } + + @Override + public boolean check(String... args) { + return args.length == 0; + } + } + + private static class GetZoneAction implements DnsAction { + + /** + * Gets details about a zone with the given name. + */ + @Override + public void run(Dns dns, String... args) { + String zoneName = args[0]; + Zone zone = dns.getZone(zoneName); + if (zone == null) { + System.out.printf("No zone with name '%s' exists.%n", zoneName); + } else { + printZone(zone); + } + } + + @Override + public String params() { + return ""; + } + + @Override + public boolean check(String... args) { + return args.length == 1; + } + } + + private static class DeleteZoneAction implements DnsAction { + + /** + * Deletes a zone with the given name. + */ + @Override + public void run(Dns dns, String... args) { + String zoneName = args[0]; + boolean deleted = dns.delete(zoneName); + if (deleted) { + System.out.printf("Zone %s was deleted.%n", zoneName); + } else { + System.out.printf("Zone %s was NOT deleted. It does not exist.%n", zoneName); + } + } + + @Override + public String params() { + return ""; + } + + @Override + public boolean check(String... args) { + return args.length == 1; + } + + } + + private static class DeleteDnsRecordAction implements DnsAction { + + /** + * Deletes a DNS record of type A from the given zone. The last parameter is ttl and it is not + * required. If ttl is not provided, a default value of 0 is used. The service requires a + * precise match (including ttl) for deleting a record. + */ + @Override + public void run(Dns dns, String... args) { + String zoneName = args[0]; + String recordName = args[1]; + String ip = args[2]; + int ttl = 0; + if (args.length > 3) { + ttl = Integer.parseInt(args[3]); + } + RecordSet recordSet = RecordSet.builder(recordName, RecordSet.Type.A) + .records(ImmutableList.of(ip)) + .ttl(ttl, TimeUnit.SECONDS) + .build(); + ChangeRequestInfo changeRequest = ChangeRequest.builder() + .delete(recordSet) + .build(); + changeRequest = dns.applyChangeRequest(zoneName, changeRequest); + System.out.printf("The request for deleting A record %s for zone %s was successfully " + + "submitted and assigned ID %s.%n", recordName, zoneName, changeRequest.id()); + System.out.print("Waiting for deletion to happen..."); + waitForChangeToFinish(dns, zoneName, changeRequest); + System.out.printf("%nThe deletion has been completed.%n"); + } + + @Override + public String params() { + return " []"; + } + + @Override + public boolean check(String... args) { + if (args.length == 4) { + // to check that it can be parsed + Integer.parseInt(args[3]); + return true; + } else { + return args.length == 3; + } + } + } + + private static class AddDnsRecordAction implements DnsAction { + + /** + * Adds a record set of type A. The last parameter is ttl and is not required. If ttl is not + * provided, a default value of 0 will be used. + */ + @Override + public void run(Dns dns, String... args) { + String zoneName = args[0]; + String recordName = args[1]; + String ip = args[2]; + int ttl = 0; + if (args.length > 3) { + ttl = Integer.parseInt(args[3]); + } + RecordSet recordSet = RecordSet.builder(recordName, RecordSet.Type.A) + .records(ImmutableList.of(ip)) + .ttl(ttl, TimeUnit.SECONDS) + .build(); + ChangeRequestInfo changeRequest = ChangeRequest.builder().add(recordSet).build(); + changeRequest = dns.applyChangeRequest(zoneName, changeRequest); + System.out.printf("The request for adding A record %s for zone %s was successfully " + + "submitted and assigned ID %s.%n", recordName, zoneName, changeRequest.id()); + System.out.print("Waiting for addition to happen..."); + waitForChangeToFinish(dns, zoneName, changeRequest); + System.out.printf("The addition has been completed.%n"); + } + + @Override + public String params() { + return " []"; + } + + @Override + public boolean check(String... args) { + if (args.length == 4) { + // to check that it can be parsed + Integer.parseInt(args[3]); + return true; + } else { + return args.length == 3; + } + } + } + + private static class ListDnsRecordsAction implements DnsAction { + + /** + * Lists all the record sets in the given zone. + */ + @Override + public void run(Dns dns, String... args) { + String zoneName = args[0]; + Iterator iterator = dns.listRecordSets(zoneName).iterateAll(); + if (iterator.hasNext()) { + System.out.printf("Record sets for zone %s:%n", zoneName); + while (iterator.hasNext()) { + RecordSet recordSet = iterator.next(); + System.out.printf("%nRecord name: %s%nTTL: %s%nRecords: %s%n", recordSet.name(), + recordSet.ttl(), Joiner.on(", ").join(recordSet.records())); + } + } else { + System.out.printf("Zone %s has no record sets records.%n", zoneName); + } + } + + @Override + public String params() { + return " records"; + } + + @Override + public boolean check(String... args) { + return args.length == 2; + } + } + + private static class ListChangesAction implements DnsAction { + + /** + * Lists all the changes for a given zone. Optionally, an order ("descending" or "ascending") + * can be specified using the last parameter. + */ + @Override + public void run(Dns dns, String... args) { + String zoneName = args[0]; + Iterator iterator; + if (args.length > 2) { + Dns.SortingOrder sortOrder = Dns.SortingOrder.valueOf(args[2].toUpperCase()); + iterator = dns.listChangeRequests(zoneName, + Dns.ChangeRequestListOption.sortOrder(sortOrder)).iterateAll(); + } else { + iterator = dns.listChangeRequests(zoneName).iterateAll(); + } + if (iterator.hasNext()) { + System.out.printf("Change requests for zone %s:%n", zoneName); + while (iterator.hasNext()) { + ChangeRequest change = iterator.next(); + System.out.printf("%nID: %s%n", change.id()); + System.out.printf("Status: %s%n", change.status()); + System.out.printf("Started: %s%n", FORMATTER.format(change.startTimeMillis())); + System.out.printf("Deletions: %s%n", Joiner.on(", ").join(change.deletions())); + System.out.printf("Additions: %s%n", Joiner.on(", ").join(change.additions())); + } + } else { + System.out.printf("Zone %s has no change requests.%n", zoneName); + } + } + + @Override + public String params() { + return " changes [descending | ascending]"; + } + + @Override + public boolean check(String... args) { + return args.length == 2 + || (args.length == 3 + && ImmutableList.of("descending", "ascending").contains(args[2].toLowerCase())); + } + } + + private static class ListAction implements DnsAction { + + /** + * Invokes a list action. If no parameter is provided, lists all zones. If zone name is the only + * parameter provided, lists both record sets and changes. Otherwise, invokes listing + * changes or zones based on the parameter provided. + */ + @Override + public void run(Dns dns, String... args) { + if (args.length == 0) { + new ListZonesAction().run(dns); + } else { + if (args.length == 1 || "records".equals(args[1])) { + new ListDnsRecordsAction().run(dns, args); + } + if (args.length == 1 || "changes".equals(args[1])) { + new ListChangesAction().run(dns, args); + } + } + } + + @Override + public boolean check(String... args) { + if (args.length == 0 || args.length == 1) { + return true; + } + if ("records".equals(args[1])) { + return new ListDnsRecordsAction().check(args); + } + if ("changes".equals(args[1])) { + return new ListChangesAction().check(args); + } + return false; + } + + @Override + public String params() { + return "[ [changes [descending | ascending] | records]]"; + } + } + + private static class GetProjectAction implements DnsAction { + + @Override + public void run(Dns dns, String... args) { + ProjectInfo project = dns.getProject(); + ProjectInfo.Quota quota = project.quota(); + System.out.printf("Project id: %s%nQuota:%n", dns.options().projectId()); + System.out.printf("\tZones: %d%n", quota.zones()); + System.out.printf("\tRecord sets per zone: %d%n", quota.rrsetsPerZone()); + System.out.printf("\tRecord sets per DNS record: %d%n", + quota.resourceRecordsPerRrset()); + System.out.printf("\tAdditions per change: %d%n", quota.rrsetAdditionsPerChange()); + System.out.printf("\tDeletions per change: %d%n", quota.rrsetDeletionsPerChange()); + System.out.printf("\tTotal data size per change: %d%n", + quota.totalRrdataSizePerChange()); + } + + @Override + public String params() { + return ""; + } + + @Override + public boolean check(String... args) { + return args.length == 0; + } + } + + static { + ACTIONS.put("create", new CreateZoneAction()); + ACTIONS.put("delete", new DeleteZoneAction()); + ACTIONS.put("get", new GetZoneAction()); + ACTIONS.put("list", new ListAction()); + ACTIONS.put("add-record", new AddDnsRecordAction()); + ACTIONS.put("delete-record", new DeleteDnsRecordAction()); + ACTIONS.put("quota", new GetProjectAction()); + } + + private static void printZone(Zone zone) { + System.out.printf("%nName: %s%n", zone.name()); + System.out.printf("ID: %s%n", zone.id()); + System.out.printf("Description: %s%n", zone.description()); + System.out.printf("Created: %s%n", FORMATTER.format(new Date(zone.creationTimeMillis()))); + System.out.printf("Name servers: %s%n", Joiner.on(", ").join(zone.nameServers())); + } + + private static ChangeRequestInfo waitForChangeToFinish(Dns dns, String zoneName, + ChangeRequestInfo request) { + ChangeRequestInfo current = request; + while (current.status().equals(ChangeRequest.Status.PENDING)) { + System.out.print("."); + try { + Thread.sleep(500); + } catch (InterruptedException e) { + System.err.println("Thread was interrupted while waiting."); + } + current = dns.getChangeRequest(zoneName, current.id()); + } + return current; + } + + private static void printUsage() { + StringBuilder actionAndParams = new StringBuilder(); + for (Map.Entry entry : ACTIONS.entrySet()) { + actionAndParams.append(System.lineSeparator()).append('\t').append(entry.getKey()); + String param = entry.getValue().params(); + if (param != null && !param.isEmpty()) { + actionAndParams.append(' ').append(param); + } + } + System.out.printf("Usage: %s [] operation *%s%n", + DnsExample.class.getSimpleName(), actionAndParams); + } + + public static void main(String... args) throws Exception { + if (args.length < 1) { + System.out.println("Missing required action"); + printUsage(); + return; + } + String projectId = null; + DnsAction action; + String actionName; + if (args.length >= 2 && !ACTIONS.containsKey(args[0])) { + actionName = args[1]; + projectId = args[0]; + args = Arrays.copyOfRange(args, 2, args.length); + } else { + actionName = args[0]; + args = Arrays.copyOfRange(args, 1, args.length); + } + action = ACTIONS.get(actionName); + if (action == null) { + System.out.printf("Unrecognized action %s.%n", actionName); + printUsage(); + return; + } + boolean valid = false; + try { + valid = action.check(args); + } catch (NumberFormatException ex) { + System.out.println("Invalid input for action '" + actionName + "'."); + System.out.println("Ttl must be an integer."); + System.out.println("Expected: " + action.params()); + return; + } catch (Exception ex) { + System.out.println("Failed to parse request."); + System.out.println("Expected: " + action.params()); + ex.printStackTrace(); + return; + } + if (valid) { + DnsOptions.Builder optionsBuilder = DnsOptions.builder(); + if (projectId != null) { + optionsBuilder.projectId(projectId); + } + Dns dns = optionsBuilder.build().service(); + action.run(dns, args); + } else { + System.out.println("Invalid input for action '" + actionName + "'"); + System.out.println("Expected: " + action.params()); + } + } +} diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateOrUpdateRecordSets.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateOrUpdateRecordSets.java new file mode 100644 index 000000000000..e3ddbb10fc0f --- /dev/null +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateOrUpdateRecordSets.java @@ -0,0 +1,74 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * EDITING INSTRUCTIONS + * This file is referenced in README's and javadoc. Any change to this file should be reflected in + * the project's README's and package-info.java. + */ + +package com.google.gcloud.examples.dns.snippets; + +import com.google.gcloud.dns.ChangeRequestInfo; +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.dns.RecordSet; +import com.google.gcloud.dns.Zone; + +import java.util.Iterator; +import java.util.concurrent.TimeUnit; + +/** + * A snippet for Google Cloud DNS showing how to create and update a resource record set. + */ +public class CreateOrUpdateRecordSets { + + public static void main(String... args) { + // Create a service object. + // The project ID and credentials will be inferred from the environment. + Dns dns = DnsOptions.defaultInstance().service(); + + // Change this to a zone name that exists within your project + String zoneName = "my-unique-zone"; + + // Get zone from the service + Zone zone = dns.getZone(zoneName); + + // Prepare a www.. type A record set with ttl of 24 hours + String ip = "12.13.14.15"; + RecordSet toCreate = RecordSet.builder("www." + zone.dnsName(), RecordSet.Type.A) + .ttl(24, TimeUnit.HOURS) + .addRecord(ip) + .build(); + + // Make a change + ChangeRequestInfo.Builder changeBuilder = ChangeRequestInfo.builder().add(toCreate); + + // Verify a www.. type A record does not exist yet. + // If it does exist, we will overwrite it with our prepared record. + Iterator recordSetIterator = zone.listRecordSets().iterateAll(); + while (recordSetIterator.hasNext()) { + RecordSet current = recordSetIterator.next(); + if (toCreate.name().equals(current.name()) && toCreate.type().equals(current.type())) { + changeBuilder.delete(current); + } + } + + // Build and apply the change request to our zone + ChangeRequestInfo changeRequest = changeBuilder.build(); + zone.applyChangeRequest(changeRequest); + } +} diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateZone.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateZone.java new file mode 100644 index 000000000000..2c2ba211bd86 --- /dev/null +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/CreateZone.java @@ -0,0 +1,51 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * EDITING INSTRUCTIONS + * This file is referenced in README's and javadoc. Any change to this file should be reflected in + * the project's README's and package-info.java. + */ + +package com.google.gcloud.examples.dns.snippets; + +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.dns.Zone; +import com.google.gcloud.dns.ZoneInfo; + +/** + * A snippet for Google Cloud DNS showing how to create a zone. You will need to change the {@code + * domainName} to a domain name, the ownership of which you should verify with Google. + */ +public class CreateZone { + + public static void main(String... args) { + // Create a service object + // The project ID and credentials will be inferred from the environment. + Dns dns = DnsOptions.defaultInstance().service(); + + // Create a zone metadata object + String zoneName = "my-unique-zone"; // Change this zone name which is unique within your project + String domainName = "someexampledomain.com."; // Change this to a domain which you own + String description = "This is a gcloud-java-dns sample zone."; + ZoneInfo zoneInfo = ZoneInfo.of(zoneName, domainName, description); + + // Create zone in Google Cloud DNS + Zone zone = dns.create(zoneInfo); + System.out.printf("Zone was created and assigned ID %s.%n", zone.id()); + } +} diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/DeleteZone.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/DeleteZone.java new file mode 100644 index 000000000000..63f26eeebb2a --- /dev/null +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/DeleteZone.java @@ -0,0 +1,88 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * EDITING INSTRUCTIONS + * This file is referenced in README's and javadoc. Any change to this file should be reflected in + * the project's README's and package-info.java. + */ + +package com.google.gcloud.examples.dns.snippets; + +import com.google.gcloud.dns.ChangeRequestInfo; +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.dns.RecordSet; + +import java.util.Iterator; + +/** + * A snippet for Google Cloud DNS showing how to delete a zone. It also shows how to list and delete + * DNS records. + */ +public class DeleteZone { + + public static void main(String... args) { + // Create a service object. + // The project ID and credentials will be inferred from the environment. + Dns dns = DnsOptions.defaultInstance().service(); + + // Change this to a zone name that exists within your project and that you want to delete. + String zoneName = "my-unique-zone"; + + // Get iterator for the existing record sets which have to be deleted before deleting the zone + Iterator recordIterator = dns.listRecordSets(zoneName).iterateAll(); + + // Make a change for deleting the records + ChangeRequestInfo.Builder changeBuilder = ChangeRequestInfo.builder(); + while (recordIterator.hasNext()) { + RecordSet current = recordIterator.next(); + // SOA and NS records cannot be deleted + if (!RecordSet.Type.SOA.equals(current.type()) && !RecordSet.Type.NS.equals(current.type())) { + changeBuilder.delete(current); + } + } + + // Build and apply the change request to our zone if it contains records to delete + ChangeRequestInfo changeRequest = changeBuilder.build(); + if (!changeRequest.deletions().isEmpty()) { + changeRequest = dns.applyChangeRequest(zoneName, changeRequest); + + // Wait for change to finish, but save data traffic by transferring only ID and status + Dns.ChangeRequestOption option = + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS); + while (ChangeRequestInfo.Status.PENDING.equals(changeRequest.status())) { + System.out.println("Waiting for change to complete. Going to sleep for 500ms..."); + try { + Thread.sleep(500); + } catch (InterruptedException e) { + System.err.println("The thread was interrupted while waiting for change request to be " + + "processed."); + } + // Update the change, but fetch only change ID and status + changeRequest = dns.getChangeRequest(zoneName, changeRequest.id(), option); + } + } + + // Delete the zone + boolean result = dns.delete(zoneName); + if (result) { + System.out.println("Zone was deleted."); + } else { + System.out.println("Zone was not deleted because it does not exist."); + } + } +} diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/ManipulateZonesAndRecordSets.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/ManipulateZonesAndRecordSets.java new file mode 100644 index 000000000000..9c9a9e77289c --- /dev/null +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/dns/snippets/ManipulateZonesAndRecordSets.java @@ -0,0 +1,158 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * EDITING INSTRUCTIONS + * This file is referenced in README's and javadoc. Any change to this file should be reflected in + * the project's README's and package-info.java. + */ + +package com.google.gcloud.examples.dns.snippets; + +import com.google.gcloud.dns.ChangeRequest; +import com.google.gcloud.dns.ChangeRequestInfo; +import com.google.gcloud.dns.Dns; +import com.google.gcloud.dns.DnsOptions; +import com.google.gcloud.dns.RecordSet; +import com.google.gcloud.dns.Zone; +import com.google.gcloud.dns.ZoneInfo; + +import java.util.Iterator; +import java.util.List; +import java.util.concurrent.TimeUnit; + +/** + * A complete snippet for Google Cloud DNS showing how to create and delete a zone. It also shows + * how to create, list and delete record sets, and how to list changes. + */ +public class ManipulateZonesAndRecordSets { + + public static void main(String... args) { + Dns dns = DnsOptions.defaultInstance().service(); + + // Create a zone metadata object + String zoneName = "my-unique-zone"; // Change this zone name which is unique within your project + String domainName = "someexampledomain.com."; // Change this to a domain which you own + String description = "This is a gcloud-java-dns sample zone."; + ZoneInfo zoneInfo = ZoneInfo.of(zoneName, domainName, description); + + // Create zone in Google Cloud DNS + Zone zone = dns.create(zoneInfo); + System.out.printf("Zone was created and assigned ID %s.%n", zone.id()); + + // Print assigned name servers + List nameServers = zone.nameServers(); + for (String nameServer : nameServers) { + System.out.println(nameServer); + } + + // Prepare a www.someexampledomain.com. type A record with ttl of 24 hours + String ip = "12.13.14.15"; + RecordSet toCreate = RecordSet.builder("www.someexampledomain.com.", RecordSet.Type.A) + .ttl(24, TimeUnit.HOURS) + .addRecord(ip) + .build(); + + // Make a change + ChangeRequestInfo.Builder changeBuilder = ChangeRequestInfo.builder().add(toCreate); + + // Verify the type A record does not exist yet. + // If it does exist, we will overwrite it with our prepared record. + Iterator recordSetIterator = zone.listRecordSets().iterateAll(); + while (recordSetIterator.hasNext()) { + RecordSet current = recordSetIterator.next(); + if (toCreate.name().equals(current.name()) && toCreate.type().equals(current.type())) { + changeBuilder.delete(current); + } + } + + // Build and apply the change request to our zone + ChangeRequestInfo changeRequest = changeBuilder.build(); + zone.applyChangeRequest(changeRequest); + + while (ChangeRequestInfo.Status.PENDING.equals(changeRequest.status())) { + try { + Thread.sleep(500L); + } catch (InterruptedException e) { + System.err.println("The thread was interrupted while waiting..."); + } + changeRequest = dns.getChangeRequest(zone.name(), changeRequest.id()); + } + System.out.println("The change request has been applied."); + + // List all your zones + Iterator zoneIterator = dns.listZones().iterateAll(); + int counter = 1; + while (zoneIterator.hasNext()) { + System.out.printf("#%d.: %s%n%n", counter, zoneIterator.next()); + counter++; + } + + // List the record sets in a particular zone + recordSetIterator = zone.listRecordSets().iterateAll(); + System.out.println(String.format("Record sets inside %s:", zone.name())); + while (recordSetIterator.hasNext()) { + System.out.println(recordSetIterator.next()); + } + + // List the change requests applied to a particular zone + Iterator changeIterator = zone.listChangeRequests().iterateAll(); + System.out.println(String.format("The history of changes in %s:", zone.name())); + while (changeIterator.hasNext()) { + System.out.println(changeIterator.next()); + } + + // Make a change for deleting the record sets + changeBuilder = ChangeRequestInfo.builder(); + while (recordSetIterator.hasNext()) { + RecordSet current = recordSetIterator.next(); + // SOA and NS records cannot be deleted + if (!RecordSet.Type.SOA.equals(current.type()) && !RecordSet.Type.NS.equals(current.type())) { + changeBuilder.delete(current); + } + } + + // Build and apply the change request to our zone if it contains records to delete + changeRequest = changeBuilder.build(); + if (!changeRequest.deletions().isEmpty()) { + changeRequest = dns.applyChangeRequest(zoneName, changeRequest); + + // Wait for change to finish, but save data traffic by transferring only ID and status + Dns.ChangeRequestOption option = + Dns.ChangeRequestOption.fields(Dns.ChangeRequestField.STATUS); + while (ChangeRequest.Status.PENDING.equals(changeRequest.status())) { + System.out.println("Waiting for change to complete. Going to sleep for 500ms..."); + try { + Thread.sleep(500); + } catch (InterruptedException e) { + System.err.println("The thread was interrupted while waiting for change request to be " + + "processed."); + } + + // Update the change, but fetch only change ID and status + changeRequest = dns.getChangeRequest(zoneName, changeRequest.id(), option); + } + } + + // Delete the zone + boolean result = dns.delete(zoneName); + if (result) { + System.out.println("Zone was deleted."); + } else { + System.out.println("Zone was not deleted because it does not exist."); + } + } +} diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/resourcemanager/snippets/ModifyPolicy.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/resourcemanager/snippets/ModifyPolicy.java new file mode 100644 index 000000000000..f97adf5b0916 --- /dev/null +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/resourcemanager/snippets/ModifyPolicy.java @@ -0,0 +1,60 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * EDITING INSTRUCTIONS + * This file is referenced in READMEs and javadoc. Any change to this file should be reflected in + * the project's READMEs and package-info.java. + */ + +package com.google.gcloud.examples.resourcemanager.snippets; + +import com.google.gcloud.Identity; +import com.google.gcloud.resourcemanager.Policy; +import com.google.gcloud.resourcemanager.Policy.ProjectRole; +import com.google.gcloud.resourcemanager.Project; +import com.google.gcloud.resourcemanager.ResourceManager; +import com.google.gcloud.resourcemanager.ResourceManagerOptions; + +/** + * A snippet for Google Cloud Resource Manager showing how to modify a project's IAM policy. + */ +public class ModifyPolicy { + + public static void main(String... args) { + // Create Resource Manager service object + // By default, credentials are inferred from the runtime environment. + ResourceManager resourceManager = ResourceManagerOptions.defaultInstance().service(); + + // Get a project from the server + String projectId = "some-project-id"; // Use an existing project's ID + Project project = resourceManager.get(projectId); + + // Get the project's policy + Policy policy = project.getPolicy(); + + // Add a viewer + Policy.Builder modifiedPolicy = policy.toBuilder(); + Identity newViewer = Identity.user(""); + modifiedPolicy.addIdentity(ProjectRole.VIEWER.value(), newViewer); + + // Write policy + Policy updatedPolicy = project.replacePolicy(modifiedPolicy.build()); + + // Print policy + System.out.printf("Updated policy for %s: %n%s%n", projectId, updatedPolicy); + } +} diff --git a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/storage/StorageExample.java b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/storage/StorageExample.java index e73cfc427129..a7260134202d 100644 --- a/gcloud-java-examples/src/main/java/com/google/gcloud/examples/storage/StorageExample.java +++ b/gcloud-java-examples/src/main/java/com/google/gcloud/examples/storage/StorageExample.java @@ -20,7 +20,6 @@ import com.google.gcloud.AuthCredentials.ServiceAccountAuthCredentials; import com.google.gcloud.ReadChannel; import com.google.gcloud.WriteChannel; -import com.google.gcloud.spi.StorageRpc.Tuple; import com.google.gcloud.storage.Blob; import com.google.gcloud.storage.BlobId; import com.google.gcloud.storage.BlobInfo; @@ -31,6 +30,7 @@ import com.google.gcloud.storage.Storage.CopyRequest; import com.google.gcloud.storage.Storage.SignUrlOption; import com.google.gcloud.storage.StorageOptions; +import com.google.gcloud.storage.spi.StorageRpc.Tuple; import java.io.FileOutputStream; import java.io.IOException; diff --git a/gcloud-java-pubsub/pom.xml b/gcloud-java-pubsub/pom.xml index 2bd755f63a8c..2014b483b17e 100644 --- a/gcloud-java-pubsub/pom.xml +++ b/gcloud-java-pubsub/pom.xml @@ -10,7 +10,7 @@ com.google.gcloud gcloud-java-pom - 0.1.4-SNAPSHOT + 0.1.5 gcloud-java-pubsub diff --git a/gcloud-java-resourcemanager/README.md b/gcloud-java-resourcemanager/README.md index 6f7d52386fc0..a2539df7adab 100644 --- a/gcloud-java-resourcemanager/README.md +++ b/gcloud-java-resourcemanager/README.md @@ -7,6 +7,7 @@ Java idiomatic client for [Google Cloud Resource Manager] (https://cloud.google. [![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) [![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-resourcemanager.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-resourcemanager.svg) [![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) - [Homepage] (https://googlecloudplatform.github.io/gcloud-java/) - [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/resourcemanager/package-summary.html) @@ -21,21 +22,21 @@ If you are using Maven, add this to your pom.xml file com.google.gcloud gcloud-java-resourcemanager - 0.1.3 + 0.1.5 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.gcloud:gcloud-java-resourcemanager:0.1.3' +compile 'com.google.gcloud:gcloud-java-resourcemanager:0.1.5' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.gcloud" % "gcloud-java-resourcemanager" % "0.1.3" +libraryDependencies += "com.google.gcloud" % "gcloud-java-resourcemanager" % "0.1.5" ``` Example Application -------------------- -[`ResourceManagerExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/resourcemanager/ResourceManagerExample.java) is a simple command line interface for the Cloud Resource Manager. Read more about using the application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/ResourceManagerExample.html). +[`ResourceManagerExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/resourcemanager/ResourceManagerExample.java) is a simple command line interface for the Cloud Resource Manager. Read more about using the application on the [`ResourceManagerExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/resourcemanager/ResourceManagerExample.html). Authentication -------------- @@ -162,9 +163,46 @@ while (projectIterator.hasNext()) { } ``` +#### Managing IAM Policies +You can edit [Google Cloud IAM](https://cloud.google.com/iam/) (Identity and Access Management) +policies on the project-level using this library as well. We recommend using the read-modify-write +pattern to make policy changes. This entails reading the project's current policy, updating it +locally, and then sending the modified policy for writing, as shown in the snippet below. First, +add these imports: + +```java +import com.google.gcloud.Identity; +import com.google.gcloud.resourcemanager.Policy; +import com.google.gcloud.resourcemanager.Policy.Role; +``` + +Assuming you have completed the steps above to create the `ResourceManager` service object and load +a project from the server, you just need to add the following code: + +```java +// Get the project's policy +Policy policy = project.getPolicy(); + +// Add a viewer +Policy.Builder modifiedPolicy = policy.toBuilder(); +Identity newViewer = Identity.user(""); +if (policy.bindings().containsKey(Role.viewer())) { + modifiedPolicy.addIdentity(Role.viewer(), newViewer); +} else { + modifiedPolicy.addBinding(Role.viewer(), newViewer); +} + +// Write policy +Policy updatedPolicy = project.replacePolicy(modifiedPolicy.build()); +``` + +Note that the policy you pass in to `replacePolicy` overwrites the original policy. For example, if +the original policy has two bindings and you call `replacePolicy` with a new policy containing only +one binding, the two original bindings are lost. + #### Complete source code -We put together all the code shown above into two programs. Both programs assume that you are +We put together all the code shown above into three programs. The programs assume that you are running from your own desktop and used the Google Cloud SDK to authenticate yourself. The first program creates a project if it does not exist. Complete source code can be found at @@ -174,6 +212,10 @@ The second program updates a project if it exists and lists all projects the use view. Complete source code can be found at [UpdateAndListProjects.java](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/resourcemanager/snippets/UpdateAndListProjects.java). +The third program modifies the IAM policy associated with a project using the read-modify-write +pattern. Complete source code can be found at +[ModifyPolicy.java](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/resourcemanager/snippets/ModifyPolicy.java) + Java Versions ------------- diff --git a/gcloud-java-resourcemanager/pom.xml b/gcloud-java-resourcemanager/pom.xml index 0e07d5afbee6..c0c48af48f1e 100644 --- a/gcloud-java-resourcemanager/pom.xml +++ b/gcloud-java-resourcemanager/pom.xml @@ -10,7 +10,7 @@ com.google.gcloud gcloud-java-pom - 0.1.4-SNAPSHOT + 0.1.6-SNAPSHOT gcloud-java-resourcemanager @@ -24,7 +24,7 @@ com.google.apis google-api-services-cloudresourcemanager - v1beta1-rev6-1.19.0 + v1beta1-rev10-1.21.0 compile @@ -33,6 +33,13 @@ + + ${project.groupId} + gcloud-java-core + ${project.version} + test-jar + test + junit junit @@ -42,7 +49,7 @@ org.easymock easymock - 3.3 + 3.4 test diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Option.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Option.java index f48c057ba049..72d62d7fc224 100644 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Option.java +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Option.java @@ -19,7 +19,7 @@ import static com.google.common.base.Preconditions.checkNotNull; import com.google.common.base.MoreObjects; -import com.google.gcloud.spi.ResourceManagerRpc; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpc; import java.io.Serializable; import java.util.Objects; diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Policy.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Policy.java new file mode 100644 index 000000000000..219d74262319 --- /dev/null +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Policy.java @@ -0,0 +1,172 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.resourcemanager; + +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Function; +import com.google.common.collect.ImmutableSet; +import com.google.common.collect.Lists; +import com.google.gcloud.IamPolicy; +import com.google.gcloud.Identity; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * An Identity and Access Management (IAM) policy for a project. IAM policies are used to specify + * access settings for Cloud Platform resources. A policy is a map of bindings. A binding assigns + * a set of identities to a role, where the identities can be user accounts, Google groups, Google + * domains, and service accounts. A role is a named list of permissions defined by IAM. Policies set + * at the project level control access both to the project and to resources associated with the + * project. + * + * @see Policy + */ +public class Policy extends IamPolicy { + + private static final long serialVersionUID = -5573557282693961850L; + + /** + * The project-level roles in an IAM policy. This enum is not an exhaustive list of all roles + * you can use in an IAM policy. You can also use service-specific roles (e.g. + * "roles/pubsub.editor"). See the Supported Cloud Platform Services page for links + * to service-specific roles. + * + * @see Supported Cloud + * Platform Services + */ + public enum ProjectRole { + + /** + * Permissions for read-only actions that preserve state. + */ + VIEWER("roles/viewer"), + + /** + * All viewer permissions and permissions for actions that modify state. + */ + EDITOR("roles/editor"), + + /** + * All editor permissions and permissions for the following actions: + *

    + *
  • Manage access control for a resource. + *
  • Set up billing (for a project). + *
+ */ + OWNER("roles/owner"); + + private final String value; + + private ProjectRole(String value) { + this.value = value; + } + + /** + * Returns the string value associated with the role. + */ + public String value() { + return value; + } + } + + /** + * Builder for an IAM Policy. + */ + public static class Builder extends IamPolicy.Builder { + + private Builder() {} + + @VisibleForTesting + Builder(Map> bindings, String etag, Integer version) { + bindings(bindings).etag(etag).version(version); + } + + @Override + public Policy build() { + return new Policy(this); + } + } + + private Policy(Builder builder) { + super(builder); + } + + public static Builder builder() { + return new Builder(); + } + + @Override + public Builder toBuilder() { + return new Builder(bindings(), etag(), version()); + } + + @Override + public String toString() { + return toPb().toString(); + } + + com.google.api.services.cloudresourcemanager.model.Policy toPb() { + com.google.api.services.cloudresourcemanager.model.Policy policyPb = + new com.google.api.services.cloudresourcemanager.model.Policy(); + List bindingPbList = + new LinkedList<>(); + for (Map.Entry> binding : bindings().entrySet()) { + com.google.api.services.cloudresourcemanager.model.Binding bindingPb = + new com.google.api.services.cloudresourcemanager.model.Binding(); + bindingPb.setRole(binding.getKey()); + bindingPb.setMembers( + Lists.transform( + new ArrayList<>(binding.getValue()), + new Function() { + @Override + public String apply(Identity identity) { + return identity.strValue(); + } + })); + bindingPbList.add(bindingPb); + } + policyPb.setBindings(bindingPbList); + policyPb.setEtag(etag()); + policyPb.setVersion(version()); + return policyPb; + } + + static Policy fromPb( + com.google.api.services.cloudresourcemanager.model.Policy policyPb) { + Map> bindings = new HashMap<>(); + for (com.google.api.services.cloudresourcemanager.model.Binding bindingPb : + policyPb.getBindings()) { + bindings.put( + bindingPb.getRole(), + ImmutableSet.copyOf( + Lists.transform( + bindingPb.getMembers(), + new Function() { + @Override + public Identity apply(String identityPb) { + return Identity.valueOf(identityPb); + } + }))); + } + return new Policy.Builder(bindings, policyPb.getEtag(), policyPb.getVersion()).build(); + } +} diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Project.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Project.java index 4d12a31274c0..bf9cf0e01a6d 100644 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Project.java +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/Project.java @@ -20,6 +20,7 @@ import java.io.IOException; import java.io.ObjectInputStream; +import java.util.List; import java.util.Map; import java.util.Objects; @@ -157,10 +158,10 @@ public Project reload() { * completes, the project is not retrievable by the {@link ResourceManager#get} and * {@link ResourceManager#list} methods. The caller must have modify permissions for this project. * - * @see Cloud - * Resource Manager delete * @throws ResourceManagerException upon failure + * @see Cloud + * Resource Manager delete */ public void delete() { resourceManager.delete(projectId()); @@ -174,10 +175,10 @@ public void delete() { * state of {@link ProjectInfo.State#DELETE_IN_PROGRESS}, the project cannot be restored. The * caller must have modify permissions for this project. * - * @see Cloud - * Resource Manager undelete * @throws ResourceManagerException upon failure (including when the project can't be restored) + * @see Cloud + * Resource Manager undelete */ public void undelete() { resourceManager.undelete(projectId()); @@ -188,16 +189,68 @@ public void undelete() { * *

The caller must have modify permissions for this project. * - * @see Cloud - * Resource Manager update * @return the Project representing the new project metadata * @throws ResourceManagerException upon failure + * @see Cloud + * Resource Manager update */ public Project replace() { return resourceManager.replace(this); } + /** + * Returns the IAM access control policy for this project. Returns {@code null} if the resource + * does not exist or if you do not have adequate permission to view the project or get the policy. + * + * @return the IAM policy for the project + * @throws ResourceManagerException upon failure + * @see + * Resource Manager getIamPolicy + */ + public Policy getPolicy() { + return resourceManager.getPolicy(projectId()); + } + + /** + * Sets the IAM access control policy for this project. Replaces any existing policy. It is + * recommended that you use the read-modify-write pattern. See code samples and important details + * of replacing policies in the documentation for {@link ResourceManager#replacePolicy}. + * + * @return the newly set IAM policy for this project + * @throws ResourceManagerException upon failure + * @see + * Resource Manager setIamPolicy + */ + public Policy replacePolicy(Policy newPolicy) { + return resourceManager.replacePolicy(projectId(), newPolicy); + } + + /** + * Returns the permissions that a caller has on this project. You typically don't call this method + * if you're using Google Cloud Platform directly to manage permissions. This method is intended + * for integration with your proprietary software, such as a customized graphical user interface. + * For example, the Cloud Platform Console tests IAM permissions internally to determine which UI + * should be available to the logged-in user. Each service that supports IAM lists the possible + * permissions; see the Supported Cloud Platform services page below for links to these + * lists. + * + * @return a list of booleans representing whether the caller has the permissions specified (in + * the order of the given permissions) + * @throws ResourceManagerException upon failure + * @see + * Resource Manager testIamPermissions + * @see Supported Cloud Platform + * Services + */ + List testPermissions(List permissions) { + return resourceManager.testPermissions(projectId(), permissions); + } + @Override public Builder toBuilder() { return new Builder(this); diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManager.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManager.java index af772dce6b60..70eeb9c8eb50 100644 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManager.java +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManager.java @@ -18,10 +18,12 @@ import com.google.common.base.Joiner; import com.google.common.collect.Sets; +import com.google.gcloud.IamPolicy; import com.google.gcloud.Page; import com.google.gcloud.Service; -import com.google.gcloud.spi.ResourceManagerRpc; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpc; +import java.util.List; import java.util.Set; /** @@ -147,8 +149,6 @@ public static ProjectListOption pageToken(String pageToken) { * *

The server can return fewer projects than requested. When there are more results than the * page size, the server will return a page token that can be used to fetch other results. - * Note: pagination is not yet supported; the server currently ignores this field and returns - * all results. */ public static ProjectListOption pageSize(int pageSize) { return new ProjectListOption(ResourceManagerRpc.Option.PAGE_SIZE, pageSize); @@ -164,25 +164,25 @@ public static ProjectListOption pageSize(int pageSize) { */ public static ProjectListOption fields(ProjectField... fields) { StringBuilder builder = new StringBuilder(); - builder.append("projects(").append(ProjectField.selector(fields)).append(")"); + builder.append("projects(").append(ProjectField.selector(fields)).append("),nextPageToken"); return new ProjectListOption(ResourceManagerRpc.Option.FIELDS, builder.toString()); } } /** - * Create a new project. + * Creates a new project. * *

Initially, the project resource is owned by its creator exclusively. The creator can later * grant permission to others to read or update the project. Several APIs are activated * automatically for the project, including Google Cloud Storage. * - * @see Cloud - * Resource Manager create * @return Project object representing the new project's metadata. The returned object will * include the following read-only fields supplied by the server: project number, lifecycle * state, and creation time. * @throws ResourceManagerException upon failure + * @see Cloud + * Resource Manager create */ Project create(ProjectInfo project); @@ -203,10 +203,10 @@ public static ProjectListOption fields(ProjectField... fields) { * completes, the project is not retrievable by the {@link ResourceManager#get} and * {@link ResourceManager#list} methods. The caller must have modify permissions for this project. * - * @see Cloud - * Resource Manager delete * @throws ResourceManagerException upon failure + * @see Cloud + * Resource Manager delete */ void delete(String projectId); @@ -216,10 +216,9 @@ public static ProjectListOption fields(ProjectField... fields) { *

Returns {@code null} if the project is not found or if the user doesn't have read * permissions for the project. * - * @see Cloud - * Resource Manager get * @throws ResourceManagerException upon failure + * @see + * Cloud Resource Manager get */ Project get(String projectId, ProjectGetOption... options); @@ -228,14 +227,13 @@ public static ProjectListOption fields(ProjectField... fields) { * *

This method returns projects in an unspecified order. New projects do not necessarily appear * at the end of the list. Use {@link ProjectListOption} to filter this list, set page size, and - * set page tokens. Note that pagination is currently not implemented by the Cloud Resource - * Manager API. + * set page tokens. * - * @see Cloud - * Resource Manager list * @return {@code Page}, a page of projects * @throws ResourceManagerException upon failure + * @see Cloud + * Resource Manager list */ Page list(ProjectListOption... options); @@ -244,11 +242,11 @@ public static ProjectListOption fields(ProjectField... fields) { * *

The caller must have modify permissions for this project. * - * @see Cloud - * Resource Manager update * @return the Project representing the new project metadata * @throws ResourceManagerException upon failure + * @see Cloud + * Resource Manager update */ Project replace(ProjectInfo newProject); @@ -260,10 +258,87 @@ public static ProjectListOption fields(ProjectField... fields) { * state of {@link ProjectInfo.State#DELETE_IN_PROGRESS}, the project cannot be restored. The * caller must have modify permissions for this project. * - * @see Cloud - * Resource Manager undelete * @throws ResourceManagerException upon failure + * @see Cloud + * Resource Manager undelete */ void undelete(String projectId); + + /** + * Returns the IAM access control policy for the specified project. Returns {@code null} if the + * resource does not exist or if you do not have adequate permission to view the project or get + * the policy. + * + * @throws ResourceManagerException upon failure + * @see + * Resource Manager getIamPolicy + */ + Policy getPolicy(String projectId); + + /** + * Sets the IAM access control policy for the specified project. Replaces any existing policy. The + * following constraints apply: + *

    + *
  • Projects currently support only user:{emailid} and serviceAccount:{emailid} + * members in a binding of a policy. + *
  • To be added as an owner, a user must be invited via Cloud Platform console and must accept + * the invitation. + *
  • Members cannot be added to more than one role in the same policy. + *
  • There must be at least one owner who has accepted the Terms of Service (ToS) agreement in + * the policy. An attempt to set a policy that removes the last ToS-accepted owner from the + * policy will fail. + *
  • Calling this method requires enabling the App Engine Admin API. + *
+ * Note: Removing service accounts from policies or changing their roles can render services + * completely inoperable. It is important to understand how the service account is being used + * before removing or updating its roles. + * + *

It is recommended that you use the read-modify-write pattern. This pattern entails reading + * the project's current policy, updating it locally, and then sending the modified policy for + * writing. Cloud IAM solves the problem of conflicting processes simultaneously attempting to + * modify a policy by using the {@link IamPolicy#etag etag} property. This property is used to + * verify whether the policy has changed since the last request. When you make a request to Cloud + * IAM with an etag value, Cloud IAM compares the etag value in the request with the existing etag + * value associated with the policy. It writes the policy only if the etag values match. If the + * etags don't match, a {@code ResourceManagerException} is thrown, denoting that the server + * aborted update. If an etag is not provided, the policy is overwritten blindly. + * + *

An example of using the read-write-modify pattern is as follows: + *

 {@code
+   * Policy currentPolicy = resourceManager.getPolicy("my-project-id");
+   * Policy modifiedPolicy =
+   *     current.toBuilder().removeIdentity(Role.VIEWER, Identity.user("user@gmail.com"));
+   * Policy newPolicy = resourceManager.replacePolicy("my-project-id", modified);
+   * }
+   * 
+ * + * @throws ResourceManagerException upon failure + * @see + * Resource Manager setIamPolicy + */ + Policy replacePolicy(String projectId, Policy newPolicy); + + /** + * Returns the permissions that a caller has on the specified project. You typically don't call + * this method if you're using Google Cloud Platform directly to manage permissions. This method + * is intended for integration with your proprietary software, such as a customized graphical user + * interface. For example, the Cloud Platform Console tests IAM permissions internally to + * determine which UI should be available to the logged-in user. Each service that supports IAM + * lists the possible permissions; see the Supported Cloud Platform services page below for + * links to these lists. + * + * @return A list of booleans representing whether the caller has the permissions specified (in + * the order of the given permissions) + * @throws ResourceManagerException upon failure + * @see + * Resource Manager testIamPermissions + * @see Supported Cloud Platform + * Services + */ + List testPermissions(String projectId, List permissions); } diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManagerImpl.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManagerImpl.java index e087caab5966..e4663cb74cb9 100644 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManagerImpl.java +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManagerImpl.java @@ -29,9 +29,10 @@ import com.google.gcloud.PageImpl; import com.google.gcloud.PageImpl.NextPageFetcher; import com.google.gcloud.RetryHelper.RetryHelperException; -import com.google.gcloud.spi.ResourceManagerRpc; -import com.google.gcloud.spi.ResourceManagerRpc.Tuple; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpc; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpc.Tuple; +import java.util.List; import java.util.Map; import java.util.concurrent.Callable; @@ -55,8 +56,8 @@ public com.google.api.services.cloudresourcemanager.model.Project call() { return resourceManagerRpc.create(project.toPb()); } }, options().retryParams(), EXCEPTION_HANDLER)); - } catch (RetryHelperException e) { - throw ResourceManagerException.translateAndThrow(e); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); } } @@ -70,8 +71,8 @@ public Void call() { return null; } }, options().retryParams(), EXCEPTION_HANDLER); - } catch (RetryHelperException e) { - throw ResourceManagerException.translateAndThrow(e); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); } } @@ -87,8 +88,8 @@ public com.google.api.services.cloudresourcemanager.model.Project call() { } }, options().retryParams(), EXCEPTION_HANDLER); return answer == null ? null : Project.fromPb(this, answer); - } catch (RetryHelperException e) { - throw ResourceManagerException.translateAndThrow(e); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); } } @@ -140,13 +141,14 @@ Iterable> call() { public Project apply( com.google.api.services.cloudresourcemanager.model.Project projectPb) { return new Project( - serviceOptions.service(), new ProjectInfo.BuilderImpl(ProjectInfo.fromPb(projectPb))); + serviceOptions.service(), + new ProjectInfo.BuilderImpl(ProjectInfo.fromPb(projectPb))); } }); return new PageImpl<>( new ProjectPageFetcher(serviceOptions, cursor, optionsMap), cursor, projects); - } catch (RetryHelperException e) { - throw ResourceManagerException.translateAndThrow(e); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); } } @@ -160,8 +162,8 @@ public com.google.api.services.cloudresourcemanager.model.Project call() { return resourceManagerRpc.replace(newProject.toPb()); } }, options().retryParams(), EXCEPTION_HANDLER)); - } catch (RetryHelperException e) { - throw ResourceManagerException.translateAndThrow(e); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); } } @@ -175,8 +177,55 @@ public Void call() { return null; } }, options().retryParams(), EXCEPTION_HANDLER); - } catch (RetryHelperException e) { - throw ResourceManagerException.translateAndThrow(e); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); + } + } + + @Override + public Policy getPolicy(final String projectId) { + try { + com.google.api.services.cloudresourcemanager.model.Policy answer = + runWithRetries( + new Callable() { + @Override + public com.google.api.services.cloudresourcemanager.model.Policy call() { + return resourceManagerRpc.getPolicy(projectId); + } + }, options().retryParams(), EXCEPTION_HANDLER); + return answer == null ? null : Policy.fromPb(answer); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); + } + } + + @Override + public Policy replacePolicy(final String projectId, final Policy newPolicy) { + try { + return Policy.fromPb(runWithRetries( + new Callable() { + @Override + public com.google.api.services.cloudresourcemanager.model.Policy call() { + return resourceManagerRpc.replacePolicy(projectId, newPolicy.toPb()); + } + }, options().retryParams(), EXCEPTION_HANDLER)); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); + } + } + + @Override + public List testPermissions(final String projectId, final List permissions) { + try { + return runWithRetries( + new Callable>() { + @Override + public List call() { + return resourceManagerRpc.testPermissions(projectId, permissions); + } + }, options().retryParams(), EXCEPTION_HANDLER); + } catch (RetryHelperException ex) { + throw ResourceManagerException.translateAndThrow(ex); } } diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManagerOptions.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManagerOptions.java index 5c0c4baf1ecb..c744864147c2 100644 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManagerOptions.java +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/ResourceManagerOptions.java @@ -18,9 +18,9 @@ import com.google.common.collect.ImmutableSet; import com.google.gcloud.ServiceOptions; -import com.google.gcloud.spi.DefaultResourceManagerRpc; -import com.google.gcloud.spi.ResourceManagerRpc; -import com.google.gcloud.spi.ResourceManagerRpcFactory; +import com.google.gcloud.resourcemanager.spi.DefaultResourceManagerRpc; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpc; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpcFactory; import java.util.Set; diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/DefaultResourceManagerRpc.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/DefaultResourceManagerRpc.java similarity index 53% rename from gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/DefaultResourceManagerRpc.java rename to gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/DefaultResourceManagerRpc.java index 61c622fa0c33..9f92ff545874 100644 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/DefaultResourceManagerRpc.java +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/DefaultResourceManagerRpc.java @@ -1,9 +1,10 @@ -package com.google.gcloud.spi; +package com.google.gcloud.resourcemanager.spi; -import static com.google.gcloud.spi.ResourceManagerRpc.Option.FIELDS; -import static com.google.gcloud.spi.ResourceManagerRpc.Option.FILTER; -import static com.google.gcloud.spi.ResourceManagerRpc.Option.PAGE_SIZE; -import static com.google.gcloud.spi.ResourceManagerRpc.Option.PAGE_TOKEN; +import static com.google.common.base.MoreObjects.firstNonNull; +import static com.google.gcloud.resourcemanager.spi.ResourceManagerRpc.Option.FIELDS; +import static com.google.gcloud.resourcemanager.spi.ResourceManagerRpc.Option.FILTER; +import static com.google.gcloud.resourcemanager.spi.ResourceManagerRpc.Option.PAGE_SIZE; +import static com.google.gcloud.resourcemanager.spi.ResourceManagerRpc.Option.PAGE_TOKEN; import static java.net.HttpURLConnection.HTTP_FORBIDDEN; import static java.net.HttpURLConnection.HTTP_NOT_FOUND; @@ -11,13 +12,22 @@ import com.google.api.client.http.HttpTransport; import com.google.api.client.json.jackson.JacksonFactory; import com.google.api.services.cloudresourcemanager.Cloudresourcemanager; +import com.google.api.services.cloudresourcemanager.model.GetIamPolicyRequest; import com.google.api.services.cloudresourcemanager.model.ListProjectsResponse; +import com.google.api.services.cloudresourcemanager.model.Policy; import com.google.api.services.cloudresourcemanager.model.Project; +import com.google.api.services.cloudresourcemanager.model.SetIamPolicyRequest; +import com.google.api.services.cloudresourcemanager.model.TestIamPermissionsRequest; +import com.google.api.services.cloudresourcemanager.model.TestIamPermissionsResponse; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; import com.google.gcloud.resourcemanager.ResourceManagerException; import com.google.gcloud.resourcemanager.ResourceManagerOptions; import java.io.IOException; +import java.util.List; import java.util.Map; +import java.util.Set; public class DefaultResourceManagerRpc implements ResourceManagerRpc { @@ -38,7 +48,7 @@ private static ResourceManagerException translate(IOException exception) { } @Override - public Project create(Project project) throws ResourceManagerException { + public Project create(Project project) { try { return resourceManager.projects().create(project).execute(); } catch (IOException ex) { @@ -47,7 +57,7 @@ public Project create(Project project) throws ResourceManagerException { } @Override - public void delete(String projectId) throws ResourceManagerException { + public void delete(String projectId) { try { resourceManager.projects().delete(projectId).execute(); } catch (IOException ex) { @@ -56,7 +66,7 @@ public void delete(String projectId) throws ResourceManagerException { } @Override - public Project get(String projectId, Map options) throws ResourceManagerException { + public Project get(String projectId, Map options) { try { return resourceManager.projects() .get(projectId) @@ -74,8 +84,7 @@ public Project get(String projectId, Map options) throws ResourceMana } @Override - public Tuple> list(Map options) - throws ResourceManagerException { + public Tuple> list(Map options) { try { ListProjectsResponse response = resourceManager.projects() .list() @@ -92,7 +101,7 @@ public Tuple> list(Map options) } @Override - public void undelete(String projectId) throws ResourceManagerException { + public void undelete(String projectId) { try { resourceManager.projects().undelete(projectId).execute(); } catch (IOException ex) { @@ -101,12 +110,58 @@ public void undelete(String projectId) throws ResourceManagerException { } @Override - public Project replace(Project project) throws ResourceManagerException { + public Project replace(Project project) { try { return resourceManager.projects().update(project.getProjectId(), project).execute(); } catch (IOException ex) { throw translate(ex); } } -} + @Override + public Policy getPolicy(String projectId) throws ResourceManagerException { + try { + return resourceManager.projects() + .getIamPolicy(projectId, new GetIamPolicyRequest()) + .execute(); + } catch (IOException ex) { + ResourceManagerException translated = translate(ex); + if (translated.code() == HTTP_FORBIDDEN) { + // Service returns permission denied if policy doesn't exist. + return null; + } else { + throw translated; + } + } + } + + @Override + public Policy replacePolicy(String projectId, Policy newPolicy) throws ResourceManagerException { + try { + return resourceManager.projects() + .setIamPolicy(projectId, new SetIamPolicyRequest().setPolicy(newPolicy)).execute(); + } catch (IOException ex) { + throw translate(ex); + } + } + + @Override + public List testPermissions(String projectId, List permissions) + throws ResourceManagerException { + try { + TestIamPermissionsResponse response = resourceManager.projects() + .testIamPermissions( + projectId, new TestIamPermissionsRequest().setPermissions(permissions)) + .execute(); + Set permissionsOwned = + ImmutableSet.copyOf(firstNonNull(response.getPermissions(), ImmutableList.of())); + ImmutableList.Builder answer = ImmutableList.builder(); + for (String p : permissions) { + answer.add(permissionsOwned.contains(p)); + } + return answer.build(); + } catch (IOException ex) { + throw translate(ex); + } + } +} diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/ResourceManagerRpc.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/ResourceManagerRpc.java new file mode 100644 index 000000000000..d6ec068a92a3 --- /dev/null +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/ResourceManagerRpc.java @@ -0,0 +1,149 @@ +/* + * Copyright 2015 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.resourcemanager.spi; + +import com.google.api.services.cloudresourcemanager.model.Policy; +import com.google.api.services.cloudresourcemanager.model.Project; +import com.google.gcloud.resourcemanager.ResourceManagerException; + +import java.util.List; +import java.util.Map; + +public interface ResourceManagerRpc { + + enum Option { + FILTER("filter"), + FIELDS("fields"), + PAGE_SIZE("pageSize"), + PAGE_TOKEN("pageToken"); + + private final String value; + + Option(String value) { + this.value = value; + } + + public String value() { + return value; + } + + @SuppressWarnings("unchecked") + T get(Map options) { + return (T) options.get(this); + } + + String getString(Map options) { + return get(options); + } + + Integer getInt(Map options) { + return get(options); + } + } + + class Tuple { + private final X x; + private final Y y; + + private Tuple(X x, Y y) { + this.x = x; + this.y = y; + } + + public static Tuple of(X x, Y y) { + return new Tuple<>(x, y); + } + + public X x() { + return x; + } + + public Y y() { + return y; + } + } + + /** + * Creates a new project. + * + * @throws ResourceManagerException upon failure + */ + Project create(Project project); + + /** + * Marks the project identified by the specified project ID for deletion. + * + * @throws ResourceManagerException upon failure + */ + void delete(String projectId); + + /** + * Retrieves the project identified by the specified project ID. Returns {@code null} if the + * project is not found or if the user doesn't have read permissions for the project. + * + * @throws ResourceManagerException upon failure + */ + Project get(String projectId, Map options); + + /** + * Lists the projects visible to the current user. + * + * @throws ResourceManagerException upon failure + */ + Tuple> list(Map options); + + /** + * Restores the project identified by the specified project ID. Undelete will only succeed if the + * project has a lifecycle state of {@code DELETE_REQUESTED} state. The caller must have modify + * permissions for this project. + * + * @throws ResourceManagerException upon failure + */ + void undelete(String projectId); + + /** + * Replaces the attributes of the project. The caller must have modify permissions for this + * project. + * + * @throws ResourceManagerException upon failure + */ + Project replace(Project project); + + /** + * Returns the IAM policy associated with a project. + * + * @throws ResourceManagerException upon failure + */ + Policy getPolicy(String projectId); + + /** + * Replaces the IAM policy associated with the given project. + * + * @throws ResourceManagerException upon failure + */ + Policy replacePolicy(String projectId, Policy newPolicy); + + /** + * Tests whether the caller has the given permissions. Returns a list of booleans corresponding to + * whether or not the user has the permission in the same position of input list. + * + * @throws ResourceManagerException upon failure + */ + List testPermissions(String projectId, List permissions); + + // TODO(ajaykannan): implement "Organization" functionality when available (issue #319) +} diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/ResourceManagerRpcFactory.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/ResourceManagerRpcFactory.java similarity index 90% rename from gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/ResourceManagerRpcFactory.java rename to gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/ResourceManagerRpcFactory.java index c2c607c0c205..4dbd1a00d4c7 100644 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/ResourceManagerRpcFactory.java +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/spi/ResourceManagerRpcFactory.java @@ -14,9 +14,10 @@ * limitations under the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.resourcemanager.spi; import com.google.gcloud.resourcemanager.ResourceManagerOptions; +import com.google.gcloud.spi.ServiceRpcFactory; /** * An interface for Resource Manager RPC factory. diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/testing/LocalResourceManagerHelper.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/testing/LocalResourceManagerHelper.java index 25c763276b3b..4d466e55a897 100644 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/testing/LocalResourceManagerHelper.java +++ b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/resourcemanager/testing/LocalResourceManagerHelper.java @@ -5,13 +5,19 @@ import static java.net.HttpURLConnection.HTTP_OK; import com.google.api.client.json.JsonFactory; +import com.google.api.services.cloudresourcemanager.model.Binding; +import com.google.api.services.cloudresourcemanager.model.Policy; import com.google.api.services.cloudresourcemanager.model.Project; +import com.google.api.services.cloudresourcemanager.model.SetIamPolicyRequest; +import com.google.api.services.cloudresourcemanager.model.TestIamPermissionsRequest; +import com.google.api.services.cloudresourcemanager.model.TestIamPermissionsResponse; import com.google.common.base.Joiner; import com.google.common.base.Objects; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import com.google.common.collect.ImmutableSet; import com.google.common.io.ByteStreams; +import com.google.gcloud.AuthCredentials; import com.google.gcloud.resourcemanager.ResourceManagerOptions; import com.sun.net.httpserver.Headers; @@ -29,21 +35,44 @@ import java.net.URISyntaxException; import java.nio.charset.StandardCharsets; import java.util.ArrayList; +import java.util.Collections; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.Random; import java.util.Set; -import java.util.concurrent.ConcurrentHashMap; +import java.util.UUID; +import java.util.concurrent.ConcurrentSkipListMap; import java.util.logging.Level; import java.util.logging.Logger; +import java.util.regex.Matcher; +import java.util.regex.Pattern; import java.util.zip.GZIPInputStream; /** * Utility to create a local Resource Manager mock for testing. * *

The mock runs in a separate thread, listening for HTTP requests on the local machine at an - * ephemeral port. + * ephemeral port. While this mock attempts to simulate the Cloud Resource Manager, there are some + * divergences in behavior. The following is a non-exhaustive list of some of those behavioral + * differences: + * + *

    + *
  • This mock assumes you have adequate permissions for any action. Related to this, + * testIamPermissions always indicates that the caller has all permissions listed in the + * request. + *
  • IAM policies are set to an empty policy with version 0 (only legacy roles supported) upon + * project creation. The actual service will not have an empty list of bindings and may also + * set your version to 1. + *
  • There is no input validation for the policy provided when replacing a policy or calling + * testIamPermissions. + *
  • In this mock, projects never move from the DELETE_REQUESTED lifecycle state to + * DELETE_IN_PROGRESS without an explicit call to the utility method + * {@link #changeLifecycleState}. Similarly, a project is never completely removed without an + * explicit call to the utility method {@link #removeProject}. + *
  • The messages in the error responses given by this mock do not necessarily match the messages + * given by the actual service. + *
*/ @SuppressWarnings("restriction") public class LocalResourceManagerHelper { @@ -56,6 +85,9 @@ public class LocalResourceManagerHelper { private static final URI BASE_CONTEXT; private static final Set SUPPORTED_COMPRESSION_ENCODINGS = ImmutableSet.of("gzip", "x-gzip"); + private static final Pattern LIST_FIELDS_PATTERN = + Pattern.compile("(.*?)projects\\((.*?)\\)(.*?)"); + private static final String[] NO_FIELDS = {}; static { try { @@ -71,7 +103,8 @@ public class LocalResourceManagerHelper { ImmutableSet.of('-', '\'', '"', ' ', '!'); private final HttpServer server; - private final ConcurrentHashMap projects = new ConcurrentHashMap<>(); + private final ConcurrentSkipListMap projects = new ConcurrentSkipListMap<>(); + private final Map policies = new HashMap<>(); private final int port; private static class Response { @@ -93,6 +126,7 @@ String body() { } private enum Error { + ABORTED(409, "global", "aborted", "ABORTED"), ALREADY_EXISTS(409, "global", "alreadyExists", "ALREADY_EXISTS"), PERMISSION_DENIED(403, "global", "forbidden", "PERMISSION_DENIED"), FAILED_PRECONDITION(400, "global", "failedPrecondition", "FAILED_PRECONDITION"), @@ -144,13 +178,7 @@ public void handle(HttpExchange exchange) { try { switch (requestMethod) { case "POST": - if (path.endsWith(":undelete")) { - response = undelete(projectIdFromUri(path)); - } else { - String requestBody = - decodeContent(exchange.getRequestHeaders(), exchange.getRequestBody()); - response = create(jsonFactory.fromString(requestBody, Project.class)); - } + response = handlePost(exchange, path); break; case "DELETE": response = delete(projectIdFromUri(path)); @@ -181,6 +209,30 @@ public void handle(HttpExchange exchange) { } } + private Response handlePost(HttpExchange exchange, String path) throws IOException { + String requestBody = decodeContent(exchange.getRequestHeaders(), exchange.getRequestBody()); + if (!path.contains(":")) { + return create(jsonFactory.fromString(requestBody, Project.class)); + } else { + switch (path.split(":", 2)[1]) { + case "undelete": + return undelete(projectIdFromUri(path)); + case "getIamPolicy": + return getPolicy(projectIdFromUri(path)); + case "setIamPolicy": + return replacePolicy(projectIdFromUri(path), + jsonFactory.fromString(requestBody, SetIamPolicyRequest.class).getPolicy()); + case "testIamPermissions": + return testPermissions(projectIdFromUri(path), + jsonFactory.fromString(requestBody, TestIamPermissionsRequest.class) + .getPermissions()); + default: + return Error.BAD_REQUEST.response( + "The server could not understand the following request URI: POST " + path); + } + } + } + private static void writeResponse(HttpExchange exchange, Response response) { exchange.getResponseHeaders().set("Content-type", "application/json; charset=UTF-8"); OutputStream outputStream = exchange.getResponseBody(); @@ -228,7 +280,7 @@ private static String[] parseFields(String query) { return null; } - private static Map parseListOptions(String query) { + private static Map parseListOptions(String query) throws IOException { Map options = new HashMap<>(); if (query != null) { String[] args = query.split("&"); @@ -236,19 +288,28 @@ private static Map parseListOptions(String query) { String[] argEntry = arg.split("="); switch (argEntry[0]) { case "fields": - // List fields are in the form "projects(field1, field2, ...)" - options.put( - "fields", - argEntry[1].substring("projects(".length(), argEntry[1].length() - 1).split(",")); + // List fields are in the form "projects(field1, field2, ...),nextPageToken" + Matcher matcher = LIST_FIELDS_PATTERN.matcher(argEntry[1]); + if (matcher.matches()) { + options.put("projectFields", matcher.group(2).split(",")); + options.put("listFields", (matcher.group(1) + matcher.group(3)).split(",")); + } else { + options.put("projectFields", NO_FIELDS); + options.put("listFields", argEntry[1].split(",")); + } break; case "filter": options.put("filter", argEntry[1].split(" ")); break; case "pageToken": - // support pageToken when Cloud Resource Manager supports this (#421) + options.put("pageToken", argEntry[1]); break; case "pageSize": - // support pageSize when Cloud Resource Manager supports this (#421) + int pageSize = Integer.parseInt(argEntry[1]); + if (pageSize < 1) { + throw new IOException("Page size must be greater than 0."); + } + options.put("pageSize", pageSize); break; } } @@ -302,7 +363,7 @@ private static boolean isValidIdOrLabel(String value, int minLength, int maxLeng return value.length() >= minLength && value.length() <= maxLength; } - Response create(Project project) { + synchronized Response create(Project project) { String customErrorMessage = checkForProjectErrors(project); if (customErrorMessage != null) { return Error.INVALID_ARGUMENT.response(customErrorMessage); @@ -314,6 +375,11 @@ Response create(Project project) { return Error.ALREADY_EXISTS.response( "A project with the same project ID (" + project.getProjectId() + ") already exists."); } + Policy emptyPolicy = new Policy() + .setBindings(Collections.emptyList()) + .setEtag(UUID.randomUUID().toString()) + .setVersion(0); + policies.put(project.getProjectId(), emptyPolicy); try { String createdProjectStr = jsonFactory.toString(project); return new Response(HTTP_OK, createdProjectStr); @@ -353,28 +419,55 @@ Response get(String projectId, String[] fields) { } Response list(Map options) { - // Use pageSize and pageToken options when Cloud Resource Manager does so (#421) List projectsSerialized = new ArrayList<>(); String[] filters = (String[]) options.get("filter"); if (filters != null && !isValidFilter(filters)) { return Error.INVALID_ARGUMENT.response("Could not parse the filter."); } - String[] fields = (String[]) options.get("fields"); - for (Project p : projects.values()) { + String[] projectFields = (String[]) options.get("projectFields"); + int count = 0; + String pageToken = (String) options.get("pageToken"); + Integer pageSize = (Integer) options.get("pageSize"); + String nextPageToken = null; + Map projectsToScan = projects; + if (pageToken != null) { + projectsToScan = projects.tailMap(pageToken); + } + for (Project p : projectsToScan.values()) { + if (pageSize != null && count >= pageSize) { + nextPageToken = p.getProjectId(); + break; + } boolean includeProject = includeProject(p, filters); if (includeProject) { + count++; try { - projectsSerialized.add(jsonFactory.toString(extractFields(p, fields))); + projectsSerialized.add(jsonFactory.toString(extractFields(p, projectFields))); } catch (IOException e) { return Error.INTERNAL_ERROR.response( "Error when serializing project " + p.getProjectId()); } } } + String[] listFields = (String[]) options.get("listFields"); StringBuilder responseBody = new StringBuilder(); - responseBody.append("{\"projects\": ["); - Joiner.on(",").appendTo(responseBody, projectsSerialized); - responseBody.append("]}"); + responseBody.append('{'); + // If fields parameter is set but no project field is selected we must return no projects. + if (!(projectFields != null && projectFields.length == 0)) { + responseBody.append("\"projects\": ["); + Joiner.on(",").appendTo(responseBody, projectsSerialized); + responseBody.append(']'); + } + if (nextPageToken != null && (listFields == null + || ImmutableSet.copyOf(listFields).contains("nextPageToken"))) { + if (responseBody.length() > 1) { + responseBody.append(','); + } + responseBody.append("\"nextPageToken\": \""); + responseBody.append(nextPageToken); + responseBody.append('"'); + } + responseBody.append('}'); return new Response(HTTP_OK, responseBody.toString()); } @@ -498,6 +591,53 @@ synchronized Response undelete(String projectId) { return response; } + synchronized Response getPolicy(String projectId) { + Policy policy = policies.get(projectId); + if (policy == null) { + return Error.PERMISSION_DENIED.response("Project " + projectId + " not found."); + } + try { + return new Response(HTTP_OK, jsonFactory.toString(policy)); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response( + "Error when serializing the IAM policy for " + projectId); + } + } + + synchronized Response replacePolicy(String projectId, Policy policy) { + Policy originalPolicy = policies.get(projectId); + if (originalPolicy == null) { + return Error.PERMISSION_DENIED.response("Error when replacing the policy for " + projectId + + " because the project was not found."); + } + String etag = policy.getEtag(); + if (etag != null && !originalPolicy.getEtag().equals(etag)) { + return Error.ABORTED.response("Policy etag mismatch when replacing the policy for project " + + projectId + ", please retry the read."); + } + policy.setEtag(UUID.randomUUID().toString()); + policy.setVersion(originalPolicy.getVersion()); + policies.put(projectId, policy); + try { + return new Response(HTTP_OK, jsonFactory.toString(policy)); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response( + "Error when serializing the policy for project " + projectId); + } + } + + synchronized Response testPermissions(String projectId, List permissions) { + if (!projects.containsKey(projectId)) { + return Error.PERMISSION_DENIED.response("Project " + projectId + " not found."); + } + try { + return new Response(HTTP_OK, + jsonFactory.toString(new TestIamPermissionsResponse().setPermissions(permissions))); + } catch (IOException e) { + return Error.INTERNAL_ERROR.response("Error when serializing permissions " + permissions); + } + } + private LocalResourceManagerHelper() { try { server = HttpServer.create(new InetSocketAddress(0), 0); @@ -509,17 +649,21 @@ private LocalResourceManagerHelper() { } /** - * Creates a LocalResourceManagerHelper object that listens to requests on the local machine. + * Creates a {@code LocalResourceManagerHelper} object that listens to requests on the local + * machine. */ public static LocalResourceManagerHelper create() { return new LocalResourceManagerHelper(); } /** - * Returns a ResourceManagerOptions instance that sets the host to use the mock server. + * Returns a {@link ResourceManagerOptions} instance that sets the host to use the mock server. */ public ResourceManagerOptions options() { - return ResourceManagerOptions.builder().host("http://localhost:" + port).build(); + return ResourceManagerOptions.builder() + .host("http://localhost:" + port) + .authCredentials(AuthCredentials.noAuth()) + .build(); } /** @@ -565,6 +709,7 @@ public synchronized boolean changeLifecycleState(String projectId, String lifecy public synchronized boolean removeProject(String projectId) { // Because this method is synchronized, any code that relies on non-atomic read/write operations // should not fail if that code is also synchronized. - return projects.remove(checkNotNull(projectId)) != null; + policies.remove(checkNotNull(projectId)); + return projects.remove(projectId) != null; } } diff --git a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/ResourceManagerRpc.java b/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/ResourceManagerRpc.java deleted file mode 100644 index 52dfc2d2368e..000000000000 --- a/gcloud-java-resourcemanager/src/main/java/com/google/gcloud/spi/ResourceManagerRpc.java +++ /dev/null @@ -1,91 +0,0 @@ -/* - * Copyright 2015 Google Inc. All Rights Reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.google.gcloud.spi; - -import com.google.api.services.cloudresourcemanager.model.Project; -import com.google.gcloud.resourcemanager.ResourceManagerException; - -import java.util.Map; - -public interface ResourceManagerRpc { - - enum Option { - FILTER("filter"), - FIELDS("fields"), - PAGE_SIZE("pageSize"), - PAGE_TOKEN("pageToken"); - - private final String value; - - Option(String value) { - this.value = value; - } - - public String value() { - return value; - } - - @SuppressWarnings("unchecked") - T get(Map options) { - return (T) options.get(this); - } - - String getString(Map options) { - return get(options); - } - - Integer getInt(Map options) { - return get(options); - } - } - - class Tuple { - private final X x; - private final Y y; - - private Tuple(X x, Y y) { - this.x = x; - this.y = y; - } - - public static Tuple of(X x, Y y) { - return new Tuple<>(x, y); - } - - public X x() { - return x; - } - - public Y y() { - return y; - } - } - - Project create(Project project) throws ResourceManagerException; - - void delete(String projectId) throws ResourceManagerException; - - Project get(String projectId, Map options) throws ResourceManagerException; - - Tuple> list(Map options) throws ResourceManagerException; - - void undelete(String projectId) throws ResourceManagerException; - - Project replace(Project project) throws ResourceManagerException; - - // TODO(ajaykannan): implement "Organization" functionality when available (issue #319) -} diff --git a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/LocalResourceManagerHelperTest.java b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/LocalResourceManagerHelperTest.java index 7eb0156d4e56..75df0ef9e3ae 100644 --- a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/LocalResourceManagerHelperTest.java +++ b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/LocalResourceManagerHelperTest.java @@ -2,23 +2,29 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotEquals; import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertNull; import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; +import com.google.api.services.cloudresourcemanager.model.Binding; +import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; +import com.google.gcloud.resourcemanager.spi.DefaultResourceManagerRpc; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpc; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpc.Tuple; import com.google.gcloud.resourcemanager.testing.LocalResourceManagerHelper; -import com.google.gcloud.spi.DefaultResourceManagerRpc; -import com.google.gcloud.spi.ResourceManagerRpc; -import com.google.gcloud.spi.ResourceManagerRpc.Tuple; import org.junit.AfterClass; import org.junit.Before; import org.junit.BeforeClass; import org.junit.Test; +import java.util.Collections; import java.util.HashMap; +import java.util.Iterator; +import java.util.List; import java.util.Map; public class LocalResourceManagerHelperTest { @@ -44,7 +50,12 @@ public class LocalResourceManagerHelperTest { .setLabels(ImmutableMap.of("k1", "v1", "k2", "v2")); private static final com.google.api.services.cloudresourcemanager.model.Project PROJECT_WITH_PARENT = - copyFrom(COMPLETE_PROJECT).setProjectId("project-with-parent-id").setParent(PARENT); + copyFrom(COMPLETE_PROJECT).setProjectId("project-with-parent-id").setParent(PARENT); + private static final List BINDINGS = ImmutableList.of( + new Binding().setRole("roles/owner").setMembers(ImmutableList.of("user:me@gmail.com")), + new Binding().setRole("roles/viewer").setMembers(ImmutableList.of("group:group@gmail.com"))); + private static final com.google.api.services.cloudresourcemanager.model.Policy POLICY = + new com.google.api.services.cloudresourcemanager.model.Policy().setBindings(BINDINGS); @BeforeClass public static void beforeClass() { @@ -91,6 +102,13 @@ public void testCreate() { assertNull(returnedProject.getParent()); assertNotNull(returnedProject.getProjectNumber()); assertNotNull(returnedProject.getCreateTime()); + com.google.api.services.cloudresourcemanager.model.Policy policy = + rpc.getPolicy(PARTIAL_PROJECT.getProjectId()); + assertEquals(Collections.emptyList(), policy.getBindings()); + assertNotNull(policy.getEtag()); + assertEquals(0, policy.getVersion().intValue()); + rpc.replacePolicy(PARTIAL_PROJECT.getProjectId(), POLICY); + assertEquals(POLICY.getBindings(), rpc.getPolicy(PARTIAL_PROJECT.getProjectId()).getBindings()); try { rpc.create(PARTIAL_PROJECT); fail("Should fail, project already exists."); @@ -98,6 +116,8 @@ public void testCreate() { assertEquals(409, e.code()); assertTrue(e.getMessage().startsWith("A project with the same project ID") && e.getMessage().endsWith("already exists.")); + assertEquals( + POLICY.getBindings(), rpc.getPolicy(PARTIAL_PROJECT.getProjectId()).getBindings()); } returnedProject = rpc.create(PROJECT_WITH_PARENT); compareReadWriteFields(PROJECT_WITH_PARENT, returnedProject); @@ -278,7 +298,7 @@ public void testGetWithOptions() { public void testList() { Tuple> projects = rpc.list(EMPTY_RPC_OPTIONS); - assertNull(projects.x()); // change this when #421 is resolved + assertNull(projects.x()); assertFalse(projects.y().iterator().hasNext()); rpc.create(COMPLETE_PROJECT); RESOURCE_MANAGER_HELPER.changeLifecycleState( @@ -297,11 +317,43 @@ public void testList() { } @Test - public void testListFieldOptions() { + public void testInvalidListPaging() { + Map rpcOptions = new HashMap<>(); + rpcOptions.put(ResourceManagerRpc.Option.PAGE_SIZE, -1); + try { + rpc.list(rpcOptions); + } catch (ResourceManagerException e) { + assertEquals("Page size must be greater than 0.", e.getMessage()); + } + } + + @Test + public void testListPaging() { Map rpcOptions = new HashMap<>(); - rpcOptions.put(ResourceManagerRpc.Option.FIELDS, "projects(projectId,name,labels)"); - rpcOptions.put(ResourceManagerRpc.Option.PAGE_TOKEN, "somePageToken"); rpcOptions.put(ResourceManagerRpc.Option.PAGE_SIZE, 1); + rpc.create(PARTIAL_PROJECT); + rpc.create(COMPLETE_PROJECT); + Tuple> projects = + rpc.list(rpcOptions); + assertNotNull(projects.x()); + Iterator iterator = + projects.y().iterator(); + compareReadWriteFields(COMPLETE_PROJECT, iterator.next()); + assertFalse(iterator.hasNext()); + rpcOptions = new HashMap<>(); + rpcOptions.put(ResourceManagerRpc.Option.PAGE_TOKEN, projects.x()); + projects = rpc.list(rpcOptions); + iterator = projects.y().iterator(); + compareReadWriteFields(PARTIAL_PROJECT, iterator.next()); + assertFalse(iterator.hasNext()); + assertNull(projects.x()); + } + + @Test + public void testListFieldOptions() { + Map rpcOptions = new HashMap<>(); + rpcOptions.put(ResourceManagerRpc.Option.FIELDS, + "projects(projectId,name,labels),nextPageToken"); rpc.create(PROJECT_WITH_PARENT); Tuple> projects = rpc.list(rpcOptions); @@ -317,6 +369,81 @@ public void testListFieldOptions() { assertNull(returnedProject.getCreateTime()); } + @Test + public void testListPageTokenFieldOptions() { + Map rpcOptions = new HashMap<>(); + rpcOptions.put(ResourceManagerRpc.Option.PAGE_SIZE, 1); + rpcOptions.put(ResourceManagerRpc.Option.FIELDS, "nextPageToken,projects(projectId,name)"); + rpc.create(PARTIAL_PROJECT); + rpc.create(COMPLETE_PROJECT); + Tuple> projects = + rpc.list(rpcOptions); + assertNotNull(projects.x()); + Iterator iterator = + projects.y().iterator(); + com.google.api.services.cloudresourcemanager.model.Project returnedProject = iterator.next(); + assertEquals(COMPLETE_PROJECT.getProjectId(), returnedProject.getProjectId()); + assertEquals(COMPLETE_PROJECT.getName(), returnedProject.getName()); + assertNull(returnedProject.getLabels()); + assertNull(returnedProject.getParent()); + assertNull(returnedProject.getProjectNumber()); + assertNull(returnedProject.getLifecycleState()); + assertNull(returnedProject.getCreateTime()); + assertFalse(iterator.hasNext()); + rpcOptions.put(ResourceManagerRpc.Option.PAGE_TOKEN, projects.x()); + projects = rpc.list(rpcOptions); + iterator = projects.y().iterator(); + returnedProject = iterator.next(); + assertEquals(PARTIAL_PROJECT.getProjectId(), returnedProject.getProjectId()); + assertEquals(PARTIAL_PROJECT.getName(), returnedProject.getName()); + assertNull(returnedProject.getLabels()); + assertNull(returnedProject.getParent()); + assertNull(returnedProject.getProjectNumber()); + assertNull(returnedProject.getLifecycleState()); + assertNull(returnedProject.getCreateTime()); + assertNull(projects.x()); + } + + @Test + public void testListNoPageTokenFieldOptions() { + Map rpcOptions = new HashMap<>(); + rpcOptions.put(ResourceManagerRpc.Option.PAGE_SIZE, 1); + rpcOptions.put(ResourceManagerRpc.Option.FIELDS, "projects(projectId,name)"); + rpc.create(PARTIAL_PROJECT); + rpc.create(COMPLETE_PROJECT); + Tuple> projects = + rpc.list(rpcOptions); + assertNull(projects.x()); + Iterator iterator = + projects.y().iterator(); + com.google.api.services.cloudresourcemanager.model.Project returnedProject = iterator.next(); + assertEquals(COMPLETE_PROJECT.getProjectId(), returnedProject.getProjectId()); + assertEquals(COMPLETE_PROJECT.getName(), returnedProject.getName()); + assertNull(returnedProject.getLabels()); + assertNull(returnedProject.getParent()); + assertNull(returnedProject.getProjectNumber()); + assertNull(returnedProject.getLifecycleState()); + assertNull(returnedProject.getCreateTime()); + assertFalse(iterator.hasNext()); + } + + @Test + public void testListPageTokenNoFieldsOptions() { + Map rpcOptions = new HashMap<>(); + rpcOptions.put(ResourceManagerRpc.Option.PAGE_SIZE, 1); + rpcOptions.put(ResourceManagerRpc.Option.FIELDS, "nextPageToken"); + rpc.create(PARTIAL_PROJECT); + rpc.create(COMPLETE_PROJECT); + Tuple> projects = + rpc.list(rpcOptions); + assertNotNull(projects.x()); + assertNull(projects.y()); + rpcOptions.put(ResourceManagerRpc.Option.PAGE_TOKEN, projects.x()); + projects = rpc.list(rpcOptions); + assertNull(projects.x()); + assertNull(projects.y()); + } + @Test public void testListFilterOptions() { Map rpcFilterOptions = new HashMap<>(); @@ -501,6 +628,58 @@ public void testUndeleteWhenDeleteInProgress() { } } + @Test + public void testGetPolicy() { + assertNull(rpc.getPolicy("nonexistent-project")); + rpc.create(PARTIAL_PROJECT); + com.google.api.services.cloudresourcemanager.model.Policy policy = + rpc.getPolicy(PARTIAL_PROJECT.getProjectId()); + assertEquals(Collections.emptyList(), policy.getBindings()); + assertNotNull(policy.getEtag()); + } + + @Test + public void testReplacePolicy() { + try { + rpc.replacePolicy("nonexistent-project", POLICY); + fail("Project doesn't exist."); + } catch (ResourceManagerException e) { + assertEquals(403, e.code()); + assertTrue(e.getMessage().contains("project was not found")); + } + rpc.create(PARTIAL_PROJECT); + com.google.api.services.cloudresourcemanager.model.Policy invalidPolicy = + new com.google.api.services.cloudresourcemanager.model.Policy().setEtag("wrong-etag"); + try { + rpc.replacePolicy(PARTIAL_PROJECT.getProjectId(), invalidPolicy); + fail("Invalid etag."); + } catch (ResourceManagerException e) { + assertEquals(409, e.code()); + assertTrue(e.getMessage().startsWith("Policy etag mismatch")); + } + String originalEtag = rpc.getPolicy(PARTIAL_PROJECT.getProjectId()).getEtag(); + com.google.api.services.cloudresourcemanager.model.Policy newPolicy = + rpc.replacePolicy(PARTIAL_PROJECT.getProjectId(), POLICY); + assertEquals(POLICY.getBindings(), newPolicy.getBindings()); + assertNotNull(newPolicy.getEtag()); + assertNotEquals(originalEtag, newPolicy.getEtag()); + } + + @Test + public void testTestPermissions() { + List permissions = ImmutableList.of("resourcemanager.projects.get"); + try { + rpc.testPermissions("nonexistent-project", permissions); + fail("Nonexistent project."); + } catch (ResourceManagerException e) { + assertEquals(403, e.code()); + assertEquals("Project nonexistent-project not found.", e.getMessage()); + } + rpc.create(PARTIAL_PROJECT); + assertEquals(ImmutableList.of(true), + rpc.testPermissions(PARTIAL_PROJECT.getProjectId(), permissions)); + } + @Test public void testChangeLifecycleStatus() { assertFalse(RESOURCE_MANAGER_HELPER.changeLifecycleState( @@ -524,8 +703,10 @@ public void testChangeLifecycleStatus() { public void testRemoveProject() { assertFalse(RESOURCE_MANAGER_HELPER.removeProject(COMPLETE_PROJECT.getProjectId())); rpc.create(COMPLETE_PROJECT); + assertNotNull(rpc.getPolicy(COMPLETE_PROJECT.getProjectId())); assertTrue(RESOURCE_MANAGER_HELPER.removeProject(COMPLETE_PROJECT.getProjectId())); assertNull(rpc.get(COMPLETE_PROJECT.getProjectId(), EMPTY_RPC_OPTIONS)); + assertNull(rpc.getPolicy(COMPLETE_PROJECT.getProjectId())); } private void compareReadWriteFields( diff --git a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/PolicyTest.java b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/PolicyTest.java new file mode 100644 index 000000000000..04826dd9540f --- /dev/null +++ b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/PolicyTest.java @@ -0,0 +1,68 @@ +/* + * Copyright 2016 Google Inc. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.gcloud.resourcemanager; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotEquals; + +import com.google.gcloud.Identity; +import com.google.gcloud.resourcemanager.Policy.ProjectRole; + +import org.junit.Test; + +public class PolicyTest { + + private static final Identity ALL_USERS = Identity.allUsers(); + private static final Identity ALL_AUTH_USERS = Identity.allAuthenticatedUsers(); + private static final Identity USER = Identity.user("abc@gmail.com"); + private static final Identity SERVICE_ACCOUNT = + Identity.serviceAccount("service-account@gmail.com"); + private static final Identity GROUP = Identity.group("group@gmail.com"); + private static final Identity DOMAIN = Identity.domain("google.com"); + private static final Policy SIMPLE_POLICY = Policy.builder() + .addIdentity(ProjectRole.OWNER.value(), USER) + .addIdentity(ProjectRole.VIEWER.value(), ALL_USERS) + .addIdentity(ProjectRole.EDITOR.value(), ALL_AUTH_USERS, DOMAIN) + .addIdentity("roles/some-role", SERVICE_ACCOUNT, GROUP) + .build(); + private static final Policy FULL_POLICY = + new Policy.Builder(SIMPLE_POLICY.bindings(), "etag", 1).build(); + + @Test + public void testIamPolicyToBuilder() { + assertEquals(FULL_POLICY, FULL_POLICY.toBuilder().build()); + assertEquals(SIMPLE_POLICY, SIMPLE_POLICY.toBuilder().build()); + } + + @Test + public void testPolicyToAndFromPb() { + assertEquals(FULL_POLICY, Policy.fromPb(FULL_POLICY.toPb())); + assertEquals(SIMPLE_POLICY, Policy.fromPb(SIMPLE_POLICY.toPb())); + } + + @Test + public void testEquals() { + Policy copy = Policy.builder() + .addIdentity(ProjectRole.OWNER.value(), USER) + .addIdentity(ProjectRole.VIEWER.value(), ALL_USERS) + .addIdentity(ProjectRole.EDITOR.value(), ALL_AUTH_USERS, DOMAIN) + .addIdentity("roles/some-role", SERVICE_ACCOUNT, GROUP) + .build(); + assertEquals(SIMPLE_POLICY, copy); + assertNotEquals(SIMPLE_POLICY, FULL_POLICY); + } +} diff --git a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/ProjectTest.java b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/ProjectTest.java index 4e239acc45ef..0f4c205dde17 100644 --- a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/ProjectTest.java +++ b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/ProjectTest.java @@ -25,12 +25,17 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNull; +import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; +import com.google.gcloud.Identity; +import com.google.gcloud.resourcemanager.Policy.ProjectRole; +import com.google.gcloud.resourcemanager.ProjectInfo.ResourceId; import org.junit.After; import org.junit.Before; import org.junit.Test; +import java.util.List; import java.util.Map; public class ProjectTest { @@ -47,6 +52,13 @@ public class ProjectTest { .createTimeMillis(CREATE_TIME_MILLIS) .state(STATE) .build(); + private static final Identity USER = Identity.user("abc@gmail.com"); + private static final Identity SERVICE_ACCOUNT = + Identity.serviceAccount("service-account@gmail.com"); + private static final Policy POLICY = Policy.builder() + .addIdentity(ProjectRole.OWNER.value(), USER) + .addIdentity(ProjectRole.EDITOR.value(), SERVICE_ACCOUNT) + .build(); private ResourceManager serviceMockReturnsOptions = createStrictMock(ResourceManager.class); private ResourceManagerOptions mockOptions = createMock(ResourceManagerOptions.class); @@ -84,12 +96,12 @@ public void testToBuilder() { @Test public void testBuilder() { - initializeExpectedProject(4); - expect(resourceManager.options()).andReturn(mockOptions).times(4); + expect(resourceManager.options()).andReturn(mockOptions).times(7); replay(resourceManager); Project.Builder builder = - new Project.Builder(new Project(resourceManager, new ProjectInfo.BuilderImpl(PROJECT_ID))); - Project project = builder.name(NAME) + new Project.Builder(new Project(resourceManager, new ProjectInfo.BuilderImpl("wrong-id"))); + Project project = builder.projectId(PROJECT_ID) + .name(NAME) .labels(LABELS) .projectNumber(PROJECT_NUMBER) .createTimeMillis(CREATE_TIME_MILLIS) @@ -102,6 +114,23 @@ public void testBuilder() { assertEquals(CREATE_TIME_MILLIS, project.createTimeMillis()); assertEquals(STATE, project.state()); assertEquals(resourceManager.options(), project.resourceManager().options()); + assertNull(project.parent()); + ResourceId parent = new ResourceId("id", "type"); + project = project.toBuilder() + .clearLabels() + .addLabel("k3", "v3") + .addLabel("k4", "v4") + .removeLabel("k4") + .parent(parent) + .build(); + assertEquals(PROJECT_ID, project.projectId()); + assertEquals(NAME, project.name()); + assertEquals(ImmutableMap.of("k3", "v3"), project.labels()); + assertEquals(PROJECT_NUMBER, project.projectNumber()); + assertEquals(CREATE_TIME_MILLIS, project.createTimeMillis()); + assertEquals(STATE, project.state()); + assertEquals(resourceManager.options(), project.resourceManager().options()); + assertEquals(parent, project.parent()); } @Test @@ -187,6 +216,39 @@ public void testReplace() { compareProjectInfos(expectedReplacedProject, actualReplacedProject); } + @Test + public void testGetPolicy() { + expect(resourceManager.options()).andReturn(mockOptions).times(1); + expect(resourceManager.getPolicy(PROJECT_ID)).andReturn(POLICY); + replay(resourceManager); + initializeProject(); + assertEquals(POLICY, project.getPolicy()); + } + + @Test + public void testReplacePolicy() { + expect(resourceManager.options()).andReturn(mockOptions).times(1); + expect(resourceManager.replacePolicy(PROJECT_ID, POLICY)).andReturn(POLICY); + replay(resourceManager); + initializeProject(); + assertEquals(POLICY, project.replacePolicy(POLICY)); + } + + @Test + public void testTestPermissions() { + List response = ImmutableList.of(true, true); + String getPermission = "resourcemanager.projects.get"; + String deletePermission = "resourcemanager.projects.delete"; + expect(resourceManager.options()).andReturn(mockOptions).times(1); + expect(resourceManager.testPermissions( + PROJECT_ID, ImmutableList.of(getPermission, deletePermission))) + .andReturn(response); + replay(resourceManager); + initializeProject(); + assertEquals( + response, project.testPermissions(ImmutableList.of(getPermission, deletePermission))); + } + private void compareProjects(Project expected, Project value) { assertEquals(expected, value); compareProjectInfos(expected, value); diff --git a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/ResourceManagerImplTest.java b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/ResourceManagerImplTest.java index 37c54718fb4a..7d52901aa372 100644 --- a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/ResourceManagerImplTest.java +++ b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/ResourceManagerImplTest.java @@ -18,21 +18,25 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotEquals; import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertNull; import static org.junit.Assert.assertSame; import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; +import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; +import com.google.gcloud.Identity; import com.google.gcloud.Page; +import com.google.gcloud.resourcemanager.Policy.ProjectRole; import com.google.gcloud.resourcemanager.ProjectInfo.ResourceId; import com.google.gcloud.resourcemanager.ResourceManager.ProjectField; import com.google.gcloud.resourcemanager.ResourceManager.ProjectGetOption; import com.google.gcloud.resourcemanager.ResourceManager.ProjectListOption; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpc; +import com.google.gcloud.resourcemanager.spi.ResourceManagerRpcFactory; import com.google.gcloud.resourcemanager.testing.LocalResourceManagerHelper; -import com.google.gcloud.spi.ResourceManagerRpc; -import com.google.gcloud.spi.ResourceManagerRpcFactory; import org.easymock.EasyMock; import org.junit.AfterClass; @@ -42,6 +46,8 @@ import org.junit.Test; import org.junit.rules.ExpectedException; +import java.util.Iterator; +import java.util.List; import java.util.Map; public class ResourceManagerImplTest { @@ -64,6 +70,12 @@ public class ResourceManagerImplTest { .parent(PARENT) .build(); private static final Map EMPTY_RPC_OPTIONS = ImmutableMap.of(); + private static final Policy POLICY = + Policy.builder() + .addIdentity(ProjectRole.OWNER.value(), Identity.user("me@gmail.com")) + .addIdentity( + ProjectRole.EDITOR.value(), Identity.serviceAccount("serviceaccount@gmail.com")) + .build(); @Rule public ExpectedException thrown = ExpectedException.none(); @@ -166,7 +178,7 @@ public void testGetWithOptions() { @Test public void testList() { Page projects = RESOURCE_MANAGER.list(); - assertFalse(projects.values().iterator().hasNext()); // TODO: change this when #421 is resolved + assertFalse(projects.values().iterator().hasNext()); RESOURCE_MANAGER.create(PARTIAL_PROJECT); RESOURCE_MANAGER.create(COMPLETE_PROJECT); for (Project p : RESOURCE_MANAGER.list().values()) { @@ -181,6 +193,22 @@ public void testList() { } } + @Test + public void testListPaging() { + RESOURCE_MANAGER.create(PARTIAL_PROJECT); + RESOURCE_MANAGER.create(COMPLETE_PROJECT); + Page page = RESOURCE_MANAGER.list(ProjectListOption.pageSize(1)); + assertNotNull(page.nextPageCursor()); + Iterator iterator = page.values().iterator(); + compareReadWriteFields(COMPLETE_PROJECT, iterator.next()); + assertFalse(iterator.hasNext()); + page = page.nextPage(); + iterator = page.values().iterator(); + compareReadWriteFields(PARTIAL_PROJECT, iterator.next()); + assertFalse(iterator.hasNext()); + assertNull(page.nextPageCursor()); + } + @Test public void testListFieldOptions() { RESOURCE_MANAGER.create(COMPLETE_PROJECT); @@ -196,6 +224,38 @@ public void testListFieldOptions() { assertSame(RESOURCE_MANAGER, returnedProject.resourceManager()); } + @Test + public void testListPagingWithFieldOptions() { + RESOURCE_MANAGER.create(PARTIAL_PROJECT); + RESOURCE_MANAGER.create(COMPLETE_PROJECT); + Page projects = RESOURCE_MANAGER.list(LIST_FIELDS, ProjectListOption.pageSize(1)); + assertNotNull(projects.nextPageCursor()); + Iterator iterator = projects.values().iterator(); + Project returnedProject = iterator.next(); + assertEquals(COMPLETE_PROJECT.projectId(), returnedProject.projectId()); + assertEquals(COMPLETE_PROJECT.name(), returnedProject.name()); + assertEquals(COMPLETE_PROJECT.labels(), returnedProject.labels()); + assertNull(returnedProject.parent()); + assertNull(returnedProject.projectNumber()); + assertNull(returnedProject.state()); + assertNull(returnedProject.createTimeMillis()); + assertSame(RESOURCE_MANAGER, returnedProject.resourceManager()); + assertFalse(iterator.hasNext()); + projects = projects.nextPage(); + iterator = projects.values().iterator(); + returnedProject = iterator.next(); + assertEquals(PARTIAL_PROJECT.projectId(), returnedProject.projectId()); + assertEquals(PARTIAL_PROJECT.name(), returnedProject.name()); + assertEquals(PARTIAL_PROJECT.labels(), returnedProject.labels()); + assertNull(returnedProject.parent()); + assertNull(returnedProject.projectNumber()); + assertNull(returnedProject.state()); + assertNull(returnedProject.createTimeMillis()); + assertSame(RESOURCE_MANAGER, returnedProject.resourceManager()); + assertFalse(iterator.hasNext()); + assertNull(projects.nextPageCursor()); + } + @Test public void testListFilterOptions() { ProjectInfo matchingProject = ProjectInfo.builder("matching-project") @@ -271,6 +331,58 @@ public void testUndelete() { } } + @Test + public void testGetPolicy() { + assertNull(RESOURCE_MANAGER.getPolicy(COMPLETE_PROJECT.projectId())); + RESOURCE_MANAGER.create(COMPLETE_PROJECT); + RESOURCE_MANAGER.replacePolicy(COMPLETE_PROJECT.projectId(), POLICY); + Policy retrieved = RESOURCE_MANAGER.getPolicy(COMPLETE_PROJECT.projectId()); + assertEquals(POLICY.bindings(), retrieved.bindings()); + assertNotNull(retrieved.etag()); + assertEquals(0, retrieved.version().intValue()); + } + + @Test + public void testReplacePolicy() { + try { + RESOURCE_MANAGER.replacePolicy("nonexistent-project", POLICY); + fail("Project doesn't exist."); + } catch (ResourceManagerException e) { + assertEquals(403, e.code()); + assertTrue(e.getMessage().endsWith("project was not found.")); + } + RESOURCE_MANAGER.create(PARTIAL_PROJECT); + Policy oldPolicy = RESOURCE_MANAGER.getPolicy(PARTIAL_PROJECT.projectId()); + RESOURCE_MANAGER.replacePolicy(PARTIAL_PROJECT.projectId(), POLICY); + try { + RESOURCE_MANAGER.replacePolicy(PARTIAL_PROJECT.projectId(), oldPolicy); + fail("Policy with an invalid etag didn't cause error."); + } catch (ResourceManagerException e) { + assertEquals(409, e.code()); + assertTrue(e.getMessage().contains("Policy etag mismatch")); + } + String originalEtag = RESOURCE_MANAGER.getPolicy(PARTIAL_PROJECT.projectId()).etag(); + Policy newPolicy = RESOURCE_MANAGER.replacePolicy(PARTIAL_PROJECT.projectId(), POLICY); + assertEquals(POLICY.bindings(), newPolicy.bindings()); + assertNotNull(newPolicy.etag()); + assertNotEquals(originalEtag, newPolicy.etag()); + } + + @Test + public void testTestPermissions() { + List permissions = ImmutableList.of("resourcemanager.projects.get"); + try { + RESOURCE_MANAGER.testPermissions("nonexistent-project", permissions); + fail("Nonexistent project"); + } catch (ResourceManagerException e) { + assertEquals(403, e.code()); + assertEquals("Project nonexistent-project not found.", e.getMessage()); + } + RESOURCE_MANAGER.create(PARTIAL_PROJECT); + assertEquals(ImmutableList.of(true), + RESOURCE_MANAGER.testPermissions(PARTIAL_PROJECT.projectId(), permissions)); + } + @Test public void testRetryableException() { ResourceManagerRpcFactory rpcFactoryMock = EasyMock.createMock(ResourceManagerRpcFactory.class); diff --git a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/SerializationTest.java b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/SerializationTest.java index 497de880254a..4bc1bcede195 100644 --- a/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/SerializationTest.java +++ b/gcloud-java-resourcemanager/src/test/java/com/google/gcloud/resourcemanager/SerializationTest.java @@ -16,24 +16,17 @@ package com.google.gcloud.resourcemanager; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotSame; - import com.google.common.collect.ImmutableMap; +import com.google.gcloud.BaseSerializationTest; +import com.google.gcloud.Identity; import com.google.gcloud.PageImpl; -import com.google.gcloud.RetryParams; - -import org.junit.Test; +import com.google.gcloud.Restorable; +import com.google.gcloud.resourcemanager.Policy.ProjectRole; -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.IOException; -import java.io.ObjectInputStream; -import java.io.ObjectOutputStream; import java.io.Serializable; import java.util.Collections; -public class SerializationTest { +public class SerializationTest extends BaseSerializationTest { private static final ResourceManager RESOURCE_MANAGER = ResourceManagerOptions.defaultInstance().service(); @@ -53,42 +46,25 @@ public class SerializationTest { ResourceManager.ProjectGetOption.fields(ResourceManager.ProjectField.NAME); private static final ResourceManager.ProjectListOption PROJECT_LIST_OPTION = ResourceManager.ProjectListOption.filter("name:*"); + private static final Policy POLICY = Policy.builder() + .addIdentity(ProjectRole.VIEWER.value(), Identity.user("abc@gmail.com")) + .build(); + private static final ResourceManagerException RESOURCE_MANAGER_EXCEPTION = + new ResourceManagerException(42, "message"); - @Test - public void testServiceOptions() throws Exception { + @Override + protected Serializable[] serializableObjects() { ResourceManagerOptions options = ResourceManagerOptions.builder().build(); - ResourceManagerOptions serializedCopy = serializeAndDeserialize(options); - assertEquals(options, serializedCopy); - options = options.toBuilder() + ResourceManagerOptions otherOptions = options.toBuilder() .projectId("some-unnecessary-project-ID") - .retryParams(RetryParams.defaultInstance()) .build(); - serializedCopy = serializeAndDeserialize(options); - assertEquals(options, serializedCopy); - } - - @Test - public void testModelAndRequests() throws Exception { - Serializable[] objects = {PARTIAL_PROJECT_INFO, FULL_PROJECT_INFO, PROJECT, PAGE_RESULT, - PROJECT_GET_OPTION, PROJECT_LIST_OPTION}; - for (Serializable obj : objects) { - Object copy = serializeAndDeserialize(obj); - assertEquals(obj, obj); - assertEquals(obj, copy); - assertNotSame(obj, copy); - assertEquals(copy, copy); - } + return new Serializable[]{PARTIAL_PROJECT_INFO, FULL_PROJECT_INFO, PROJECT, PAGE_RESULT, + PROJECT_GET_OPTION, PROJECT_LIST_OPTION, POLICY, RESOURCE_MANAGER_EXCEPTION, options, + otherOptions}; } - @SuppressWarnings("unchecked") - private T serializeAndDeserialize(T obj) throws IOException, ClassNotFoundException { - ByteArrayOutputStream bytes = new ByteArrayOutputStream(); - try (ObjectOutputStream output = new ObjectOutputStream(bytes)) { - output.writeObject(obj); - } - try (ObjectInputStream input = - new ObjectInputStream(new ByteArrayInputStream(bytes.toByteArray()))) { - return (T) input.readObject(); - } + @Override + protected Restorable[] restorableObjects() { + return null; } } diff --git a/gcloud-java-storage/README.md b/gcloud-java-storage/README.md index 554f22b38e7f..0ee05b31c10c 100644 --- a/gcloud-java-storage/README.md +++ b/gcloud-java-storage/README.md @@ -7,6 +7,7 @@ Java idiomatic client for [Google Cloud Storage] (https://cloud.google.com/stora [![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) [![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-storage.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java-storage.svg) [![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) - [Homepage] (https://googlecloudplatform.github.io/gcloud-java/) - [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs/index.html?com/google/gcloud/storage/package-summary.html) @@ -21,22 +22,22 @@ If you are using Maven, add this to your pom.xml file com.google.gcloud gcloud-java-storage - 0.1.3 + 0.1.5 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.gcloud:gcloud-java-storage:0.1.3' +compile 'com.google.gcloud:gcloud-java-storage:0.1.5' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.gcloud" % "gcloud-java-storage" % "0.1.3" +libraryDependencies += "com.google.gcloud" % "gcloud-java-storage" % "0.1.5" ``` Example Application ------------------- -[`StorageExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/storage/StorageExample.java) is a simple command line interface that provides some of Cloud Storage's functionality. Read more about using the application on the [`gcloud-java-examples` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/StorageExample.html). +[`StorageExample`](../gcloud-java-examples/src/main/java/com/google/gcloud/examples/storage/StorageExample.java) is a simple command line interface that provides some of Cloud Storage's functionality. Read more about using the application on the [`StorageExample` docs page](http://googlecloudplatform.github.io/gcloud-java/apidocs/?com/google/gcloud/examples/storage/StorageExample.html). Authentication -------------- diff --git a/gcloud-java-storage/pom.xml b/gcloud-java-storage/pom.xml index 6b4755d90041..16427d50de3a 100644 --- a/gcloud-java-storage/pom.xml +++ b/gcloud-java-storage/pom.xml @@ -10,7 +10,7 @@ com.google.gcloud gcloud-java-pom - 0.1.4-SNAPSHOT + 0.1.6-SNAPSHOT gcloud-java-storage @@ -24,19 +24,26 @@ com.google.apis google-api-services-storage - v1-rev33-1.20.0 + v1-rev62-1.21.0 compile - - com.google.guava - guava-jdk5 - + + com.google.guava + guava-jdk5 + com.google.api-client google-api-client + + ${project.groupId} + gcloud-java-core + ${project.version} + test-jar + test + junit junit @@ -46,7 +53,7 @@ org.easymock easymock - 3.3 + 3.4 test diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Blob.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Blob.java index ec8ab361328e..b6f668dada82 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Blob.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Blob.java @@ -22,14 +22,15 @@ import com.google.api.services.storage.model.StorageObject; import com.google.common.base.Function; +import com.google.gcloud.AuthCredentials; import com.google.gcloud.ReadChannel; import com.google.gcloud.WriteChannel; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpc.Tuple; import com.google.gcloud.storage.Storage.BlobTargetOption; import com.google.gcloud.storage.Storage.BlobWriteOption; import com.google.gcloud.storage.Storage.CopyRequest; import com.google.gcloud.storage.Storage.SignUrlOption; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc.Tuple; import java.io.IOException; import java.io.ObjectInputStream; @@ -49,7 +50,7 @@ * {@link BlobInfo}. *

*/ -public final class Blob extends BlobInfo { +public class Blob extends BlobInfo { private static final long serialVersionUID = -6806832496717441434L; @@ -290,6 +291,12 @@ Builder updateTime(Long updateTime) { return this; } + @Override + Builder isDirectory(boolean isDirectory) { + infoBuilder.isDirectory(isDirectory); + return this; + } + @Override public Blob build() { return new Blob(storage, infoBuilder); @@ -351,8 +358,10 @@ public Blob reload(BlobSourceOption... options) { *

* *

Example usage of replacing blob's metadata: - *

    {@code blob.toBuilder().metadata(null).build().update();}
-   *    {@code blob.toBuilder().metadata(newMetadata).build().update();}
+   * 
 {@code
+   * blob.toBuilder().metadata(null).build().update();
+   * blob.toBuilder().metadata(newMetadata).build().update();
+   * }
    * 
* * @param options update options @@ -445,16 +454,48 @@ public WriteChannel writer(BlobWriteOption... options) { } /** - * Generates a signed URL for this blob. If you want to allow access to for a fixed amount of time - * for this blob, you can use this method to generate a URL that is only valid within a certain - * time period. This is particularly useful if you don't want publicly accessible blobs, but don't - * want to require users to explicitly log in. + * Generates a signed URL for this blob. If you want to allow access for a fixed amount of time to + * this blob, you can use this method to generate a URL that is only valid within a certain time + * period. This is particularly useful if you don't want publicly accessible blobs, but also don't + * want to require users to explicitly log in. Signing a URL requires a service account and its + * associated private key. If a {@link AuthCredentials.ServiceAccountAuthCredentials} was passed + * to {@link StorageOptions.Builder#authCredentials(AuthCredentials)} or the default credentials + * are being used and the environment variable {@code GOOGLE_APPLICATION_CREDENTIALS} is set, then + * {@code signUrl} will use that service account and associated key to sign the URL. If the + * credentials passed to {@link StorageOptions} do not expose a private key (this is the case for + * App Engine credentials, Compute Engine credentials and Google Cloud SDK credentials) then + * {@code signUrl} will throw an {@link IllegalArgumentException} unless a service account with + * associated key is passed using the {@code SignUrlOption.serviceAccount()} option. The service + * account and private key passed with {@code SignUrlOption.serviceAccount()} have priority over + * any credentials set with {@link StorageOptions.Builder#authCredentials(AuthCredentials)}. + * + *

Example usage of creating a signed URL that is valid for 2 weeks, using the default + * credentials for signing the URL: + *

 {@code
+   * blob.signUrl(14, TimeUnit.DAYS);
+   * }
+ * + *

Example usage of creating a signed URL passing the {@code SignUrlOption.serviceAccount()} + * option, that will be used for signing the URL: + *

 {@code
+   * blob.signUrl(14, TimeUnit.DAYS, SignUrlOption.serviceAccount(
+   *     AuthCredentials.createForJson(new FileInputStream("/path/to/key.json"))));
+   * }
* * @param duration time until the signed URL expires, expressed in {@code unit}. The finer * granularity supported is 1 second, finer granularities will be truncated * @param unit time unit of the {@code duration} parameter * @param options optional URL signing options * @return a signed URL for this bucket and the specified options + * @throws IllegalArgumentException if + * {@link SignUrlOption#serviceAccount(AuthCredentials.ServiceAccountAuthCredentials)} was not + * used and no service account was provided to {@link StorageOptions} + * @throws IllegalArgumentException if the key associated to the provided service account is + * invalid + * @throws IllegalArgumentException if {@link SignUrlOption#withMd5()} option is used and + * {@link #md5()} is {@code null} + * @throws IllegalArgumentException if {@link SignUrlOption#withContentType()} option is used and + * {@link #contentType()} is {@code null} * @see Signed-URLs */ public URL signUrl(long duration, TimeUnit unit, SignUrlOption... options) { @@ -474,13 +515,13 @@ public Builder toBuilder() { } @Override - public boolean equals(Object obj) { + public final boolean equals(Object obj) { return obj instanceof Blob && Objects.equals(toPb(), ((Blob) obj).toPb()) && Objects.equals(options, ((Blob) obj).options); } @Override - public int hashCode() { + public final int hashCode() { return Objects.hash(super.hashCode(), options); } diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobInfo.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobInfo.java index 54fabe87d766..cf509c8f0961 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobInfo.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobInfo.java @@ -78,6 +78,7 @@ public StorageObject apply(BlobInfo blobInfo) { private final String contentDisposition; private final String contentLanguage; private final Integer componentCount; + private final boolean isDirectory; /** * This class is meant for internal use only. Users are discouraged from using this class. @@ -187,6 +188,8 @@ public abstract static class Builder { abstract Builder updateTime(Long updateTime); + abstract Builder isDirectory(boolean isDirectory); + /** * Creates a {@code BlobInfo} object. */ @@ -215,6 +218,7 @@ static final class BuilderImpl extends Builder { private Long metageneration; private Long deleteTime; private Long updateTime; + private Boolean isDirectory; BuilderImpl(BlobId blobId) { this.blobId = blobId; @@ -241,6 +245,7 @@ static final class BuilderImpl extends Builder { metageneration = blobInfo.metageneration; deleteTime = blobInfo.deleteTime; updateTime = blobInfo.updateTime; + isDirectory = blobInfo.isDirectory; } @Override @@ -364,6 +369,12 @@ Builder updateTime(Long updateTime) { return this; } + @Override + Builder isDirectory(boolean isDirectory) { + this.isDirectory = isDirectory; + return this; + } + @Override public BlobInfo build() { checkNotNull(blobId); @@ -392,6 +403,7 @@ public BlobInfo build() { metageneration = builder.metageneration; deleteTime = builder.deleteTime; updateTime = builder.updateTime; + isDirectory = firstNonNull(builder.isDirectory, Boolean.FALSE); } /** @@ -588,6 +600,18 @@ public Long updateTime() { return updateTime; } + /** + * Returns {@code true} if the current blob represents a directory. This can only happen if the + * blob is returned by {@link Storage#list(String, Storage.BlobListOption...)} when the + * {@link Storage.BlobListOption#currentDirectory()} option is used. When this is the case only + * {@link #blobId()} and {@link #size()} are set for the current blob: {@link BlobId#name()} ends + * with the '/' character, {@link BlobId#generation()} returns {@code null} and {@link #size()} is + * {@code 0}. + */ + public boolean isDirectory() { + return isDirectory; + } + /** * Returns a builder for the current blob. */ @@ -761,6 +785,9 @@ public Acl apply(ObjectAccessControl objectAccessControl) { } })); } + if (storageObject.containsKey("isDirectory")) { + builder.isDirectory(Boolean.TRUE); + } return builder.build(); } } diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobReadChannel.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobReadChannel.java index 121f2eb63589..f9c6f912563d 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobReadChannel.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobReadChannel.java @@ -23,12 +23,13 @@ import com.google.gcloud.ReadChannel; import com.google.gcloud.RestorableState; import com.google.gcloud.RetryHelper; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpc.Tuple; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc.Tuple; import java.io.IOException; import java.io.Serializable; import java.nio.ByteBuffer; +import java.nio.channels.ClosedChannelException; import java.util.Map; import java.util.Objects; import java.util.concurrent.Callable; @@ -55,7 +56,7 @@ class BlobReadChannel implements ReadChannel { private byte[] buffer; BlobReadChannel(StorageOptions serviceOptions, BlobId blob, - Map requestOptions) { + Map requestOptions) { this.serviceOptions = serviceOptions; this.blob = blob; this.requestOptions = requestOptions; @@ -91,9 +92,9 @@ public void close() { } } - private void validateOpen() throws IOException { + private void validateOpen() throws ClosedChannelException { if (!isOpen) { - throw new IOException("stream is closed"); + throw new ClosedChannelException(); } } @@ -126,7 +127,7 @@ public Tuple call() { return storageRpc.read(storageObject, requestOptions, position, toRead); } }, serviceOptions.retryParams(), StorageImpl.EXCEPTION_HANDLER); - if (lastEtag != null && !Objects.equals(result.x(), lastEtag)) { + if (result.y().length > 0 && lastEtag != null && !Objects.equals(result.x(), lastEtag)) { StringBuilder messageBuilder = new StringBuilder(); messageBuilder.append("Blob ").append(blob).append(" was updated while reading"); throw new StorageException(0, messageBuilder.toString()); diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobWriteChannel.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobWriteChannel.java index d1d12ec77638..30b0ec870f51 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobWriteChannel.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BlobWriteChannel.java @@ -23,7 +23,7 @@ import com.google.gcloud.RestorableState; import com.google.gcloud.RetryHelper; import com.google.gcloud.WriteChannel; -import com.google.gcloud.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc; import java.util.Map; diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Bucket.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Bucket.java index c438f497730e..e44bd60d785c 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Bucket.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Bucket.java @@ -16,26 +16,29 @@ package com.google.gcloud.storage; +import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.gcloud.storage.Bucket.BucketSourceOption.toGetOptions; import static com.google.gcloud.storage.Bucket.BucketSourceOption.toSourceOptions; -import com.google.common.base.MoreObjects; +import com.google.common.base.Function; +import com.google.common.collect.Lists; +import com.google.common.collect.Sets; import com.google.gcloud.Page; -import com.google.gcloud.spi.StorageRpc; import com.google.gcloud.storage.Storage.BlobGetOption; -import com.google.gcloud.storage.Storage.BlobTargetOption; -import com.google.gcloud.storage.Storage.BlobWriteOption; import com.google.gcloud.storage.Storage.BucketTargetOption; +import com.google.gcloud.storage.spi.StorageRpc; import java.io.IOException; import java.io.InputStream; import java.io.ObjectInputStream; +import java.io.Serializable; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List; import java.util.Objects; +import java.util.Set; /** * A Google cloud storage bucket. @@ -64,7 +67,7 @@ private BucketSourceOption(StorageRpc.Option rpcOption) { super(rpcOption, null); } - private Storage.BucketSourceOption toSourceOptions(BucketInfo bucketInfo) { + private Storage.BucketSourceOption toSourceOption(BucketInfo bucketInfo) { switch (rpcOption()) { case IF_METAGENERATION_MATCH: return Storage.BucketSourceOption.metagenerationMatch(bucketInfo.metageneration()); @@ -108,7 +111,7 @@ static Storage.BucketSourceOption[] toSourceOptions(BucketInfo bucketInfo, new Storage.BucketSourceOption[options.length]; int index = 0; for (BucketSourceOption option : options) { - convertedOptions[index++] = option.toSourceOptions(bucketInfo); + convertedOptions[index++] = option.toSourceOption(bucketInfo); } return convertedOptions; } @@ -124,6 +127,287 @@ static Storage.BucketGetOption[] toGetOptions(BucketInfo bucketInfo, } } + /** + * Class for specifying blob target options when {@code Bucket} methods are used. + */ + public static class BlobTargetOption extends Option { + + private static final Function TO_ENUM = + new Function() { + @Override + public StorageRpc.Option apply(BlobTargetOption blobTargetOption) { + return blobTargetOption.rpcOption(); + } + }; + private static final long serialVersionUID = 8345296337342509425L; + + private BlobTargetOption(StorageRpc.Option rpcOption, Object value) { + super(rpcOption, value); + } + + private StorageRpc.Tuple toTargetOption(BlobInfo blobInfo) { + BlobId blobId = blobInfo.blobId(); + switch (rpcOption()) { + case PREDEFINED_ACL: + return StorageRpc.Tuple.of(blobInfo, + Storage.BlobTargetOption.predefinedAcl((Storage.PredefinedAcl) value())); + case IF_GENERATION_MATCH: + blobId = BlobId.of(blobId.bucket(), blobId.name(), (Long) value()); + return StorageRpc.Tuple.of(blobInfo.toBuilder().blobId(blobId).build(), + Storage.BlobTargetOption.generationMatch()); + case IF_GENERATION_NOT_MATCH: + blobId = BlobId.of(blobId.bucket(), blobId.name(), (Long) value()); + return StorageRpc.Tuple.of(blobInfo.toBuilder().blobId(blobId).build(), + Storage.BlobTargetOption.generationNotMatch()); + case IF_METAGENERATION_MATCH: + return StorageRpc.Tuple.of(blobInfo.toBuilder().metageneration((Long) value()).build(), + Storage.BlobTargetOption.metagenerationMatch()); + case IF_METAGENERATION_NOT_MATCH: + return StorageRpc.Tuple.of(blobInfo.toBuilder().metageneration((Long) value()).build(), + Storage.BlobTargetOption.metagenerationNotMatch()); + default: + throw new AssertionError("Unexpected enum value"); + } + } + + /** + * Returns an option for specifying blob's predefined ACL configuration. + */ + public static BlobTargetOption predefinedAcl(Storage.PredefinedAcl acl) { + return new BlobTargetOption(StorageRpc.Option.PREDEFINED_ACL, acl); + } + + /** + * Returns an option that causes an operation to succeed only if the target blob does not exist. + * This option can not be provided together with {@link #generationMatch(long)} or + * {@link #generationNotMatch(long)}. + */ + public static BlobTargetOption doesNotExist() { + return new BlobTargetOption(StorageRpc.Option.IF_GENERATION_MATCH, 0L); + } + + /** + * Returns an option for blob's data generation match. If this option is used the request will + * fail if generation does not match the provided value. This option can not be provided + * together with {@link #generationNotMatch(long)} or {@link #doesNotExist()}. + */ + public static BlobTargetOption generationMatch(long generation) { + return new BlobTargetOption(StorageRpc.Option.IF_GENERATION_MATCH, generation); + } + + /** + * Returns an option for blob's data generation mismatch. If this option is used the request + * will fail if blob's generation matches the provided value. This option can not be provided + * together with {@link #generationMatch(long)} or {@link #doesNotExist()}. + */ + public static BlobTargetOption generationNotMatch(long generation) { + return new BlobTargetOption(StorageRpc.Option.IF_GENERATION_NOT_MATCH, generation); + } + + /** + * Returns an option for blob's metageneration match. If this option is used the request will + * fail if metageneration does not match the provided value. This option can not be provided + * together with {@link #metagenerationNotMatch(long)}. + */ + public static BlobTargetOption metagenerationMatch(long metageneration) { + return new BlobTargetOption(StorageRpc.Option.IF_METAGENERATION_MATCH, metageneration); + } + + /** + * Returns an option for blob's metageneration mismatch. If this option is used the request will + * fail if metageneration matches the provided value. This option can not be provided together + * with {@link #metagenerationMatch(long)}. + */ + public static BlobTargetOption metagenerationNotMatch(long metageneration) { + return new BlobTargetOption(StorageRpc.Option.IF_METAGENERATION_NOT_MATCH, metageneration); + } + + static StorageRpc.Tuple toTargetOptions( + BlobInfo info, BlobTargetOption... options) { + Set optionSet = + Sets.immutableEnumSet(Lists.transform(Arrays.asList(options), TO_ENUM)); + checkArgument(!(optionSet.contains(StorageRpc.Option.IF_METAGENERATION_NOT_MATCH) + && optionSet.contains(StorageRpc.Option.IF_METAGENERATION_MATCH)), + "metagenerationMatch and metagenerationNotMatch options can not be both provided"); + checkArgument(!(optionSet.contains(StorageRpc.Option.IF_GENERATION_NOT_MATCH) + && optionSet.contains(StorageRpc.Option.IF_GENERATION_MATCH)), + "Only one option of generationMatch, doesNotExist or generationNotMatch can be provided"); + Storage.BlobTargetOption[] convertedOptions = new Storage.BlobTargetOption[options.length]; + BlobInfo targetInfo = info; + int index = 0; + for (BlobTargetOption option : options) { + StorageRpc.Tuple target = + option.toTargetOption(targetInfo); + targetInfo = target.x(); + convertedOptions[index++] = target.y(); + } + return StorageRpc.Tuple.of(targetInfo, convertedOptions); + } + } + + /** + * Class for specifying blob write options when {@code Bucket} methods are used. + */ + public static class BlobWriteOption implements Serializable { + + private static final Function TO_ENUM = + new Function() { + @Override + public Storage.BlobWriteOption.Option apply(BlobWriteOption blobWriteOption) { + return blobWriteOption.option; + } + }; + private static final long serialVersionUID = 4722190734541993114L; + + private final Storage.BlobWriteOption.Option option; + private final Object value; + + private StorageRpc.Tuple toWriteOption(BlobInfo blobInfo) { + BlobId blobId = blobInfo.blobId(); + switch (option) { + case PREDEFINED_ACL: + return StorageRpc.Tuple.of(blobInfo, + Storage.BlobWriteOption.predefinedAcl((Storage.PredefinedAcl) value)); + case IF_GENERATION_MATCH: + blobId = BlobId.of(blobId.bucket(), blobId.name(), (Long) value); + return StorageRpc.Tuple.of(blobInfo.toBuilder().blobId(blobId).build(), + Storage.BlobWriteOption.generationMatch()); + case IF_GENERATION_NOT_MATCH: + blobId = BlobId.of(blobId.bucket(), blobId.name(), (Long) value); + return StorageRpc.Tuple.of(blobInfo.toBuilder().blobId(blobId).build(), + Storage.BlobWriteOption.generationNotMatch()); + case IF_METAGENERATION_MATCH: + return StorageRpc.Tuple.of(blobInfo.toBuilder().metageneration((Long) value).build(), + Storage.BlobWriteOption.metagenerationMatch()); + case IF_METAGENERATION_NOT_MATCH: + return StorageRpc.Tuple.of(blobInfo.toBuilder().metageneration((Long) value).build(), + Storage.BlobWriteOption.metagenerationNotMatch()); + case IF_MD5_MATCH: + return StorageRpc.Tuple.of(blobInfo.toBuilder().md5((String) value).build(), + Storage.BlobWriteOption.md5Match()); + case IF_CRC32C_MATCH: + return StorageRpc.Tuple.of(blobInfo.toBuilder().crc32c((String) value).build(), + Storage.BlobWriteOption.crc32cMatch()); + default: + throw new AssertionError("Unexpected enum value"); + } + } + + private BlobWriteOption(Storage.BlobWriteOption.Option option, Object value) { + this.option = option; + this.value = value; + } + + @Override + public int hashCode() { + return Objects.hash(option, value); + } + + @Override + public boolean equals(Object obj) { + if (obj == null) { + return false; + } + if (!(obj instanceof BlobWriteOption)) { + return false; + } + final BlobWriteOption other = (BlobWriteOption) obj; + return this.option == other.option && Objects.equals(this.value, other.value); + } + + /** + * Returns an option for specifying blob's predefined ACL configuration. + */ + public static BlobWriteOption predefinedAcl(Storage.PredefinedAcl acl) { + return new BlobWriteOption(Storage.BlobWriteOption.Option.PREDEFINED_ACL, acl); + } + + /** + * Returns an option that causes an operation to succeed only if the target blob does not exist. + * This option can not be provided together with {@link #generationMatch(long)} or + * {@link #generationNotMatch(long)}. + */ + public static BlobWriteOption doesNotExist() { + return new BlobWriteOption(Storage.BlobWriteOption.Option.IF_GENERATION_MATCH, 0L); + } + + /** + * Returns an option for blob's data generation match. If this option is used the request will + * fail if generation does not match the provided value. This option can not be provided + * together with {@link #generationNotMatch(long)} or {@link #doesNotExist()}. + */ + public static BlobWriteOption generationMatch(long generation) { + return new BlobWriteOption(Storage.BlobWriteOption.Option.IF_GENERATION_MATCH, generation); + } + + /** + * Returns an option for blob's data generation mismatch. If this option is used the request + * will fail if generation matches the provided value. This option can not be provided + * together with {@link #generationMatch(long)} or {@link #doesNotExist()}. + */ + public static BlobWriteOption generationNotMatch(long generation) { + return new BlobWriteOption(Storage.BlobWriteOption.Option.IF_GENERATION_NOT_MATCH, + generation); + } + + /** + * Returns an option for blob's metageneration match. If this option is used the request will + * fail if metageneration does not match the provided value. This option can not be provided + * together with {@link #metagenerationNotMatch(long)}. + */ + public static BlobWriteOption metagenerationMatch(long metageneration) { + return new BlobWriteOption(Storage.BlobWriteOption.Option.IF_METAGENERATION_MATCH, + metageneration); + } + + /** + * Returns an option for blob's metageneration mismatch. If this option is used the request will + * fail if metageneration matches the provided value. This option can not be provided together + * with {@link #metagenerationMatch(long)}. + */ + public static BlobWriteOption metagenerationNotMatch(long metageneration) { + return new BlobWriteOption(Storage.BlobWriteOption.Option.IF_METAGENERATION_NOT_MATCH, + metageneration); + } + + /** + * Returns an option for blob's data MD5 hash match. If this option is used the request will + * fail if blobs' data MD5 hash does not match the provided value. + */ + public static BlobWriteOption md5Match(String md5) { + return new BlobWriteOption(Storage.BlobWriteOption.Option.IF_MD5_MATCH, md5); + } + + /** + * Returns an option for blob's data CRC32C checksum match. If this option is used the request + * will fail if blobs' data CRC32C checksum does not match the provided value. + */ + public static BlobWriteOption crc32cMatch(String crc32c) { + return new BlobWriteOption(Storage.BlobWriteOption.Option.IF_CRC32C_MATCH, crc32c); + } + + static StorageRpc.Tuple toWriteOptions( + BlobInfo info, BlobWriteOption... options) { + Set optionSet = + Sets.immutableEnumSet(Lists.transform(Arrays.asList(options), TO_ENUM)); + checkArgument(!(optionSet.contains(Storage.BlobWriteOption.Option.IF_METAGENERATION_NOT_MATCH) + && optionSet.contains(Storage.BlobWriteOption.Option.IF_METAGENERATION_MATCH)), + "metagenerationMatch and metagenerationNotMatch options can not be both provided"); + checkArgument(!(optionSet.contains(Storage.BlobWriteOption.Option.IF_GENERATION_NOT_MATCH) + && optionSet.contains(Storage.BlobWriteOption.Option.IF_GENERATION_MATCH)), + "Only one option of generationMatch, doesNotExist or generationNotMatch can be provided"); + Storage.BlobWriteOption[] convertedOptions = new Storage.BlobWriteOption[options.length]; + BlobInfo writeInfo = info; + int index = 0; + for (BlobWriteOption option : options) { + StorageRpc.Tuple write = option.toWriteOption(writeInfo); + writeInfo = write.x(); + convertedOptions[index++] = write.y(); + } + return StorageRpc.Tuple.of(writeInfo, convertedOptions); + } + } + /** * Builder for {@code Bucket}. */ @@ -348,16 +632,16 @@ public List get(String blobName1, String blobName2, String... blobNames) { * * @param blob a blob name * @param content the blob content - * @param contentType the blob content type. If {@code null} then - * {@value com.google.gcloud.storage.Storage#DEFAULT_CONTENT_TYPE} is used. + * @param contentType the blob content type * @param options options for blob creation * @return a complete blob information * @throws StorageException upon failure */ public Blob create(String blob, byte[] content, String contentType, BlobTargetOption... options) { - BlobInfo blobInfo = BlobInfo.builder(BlobId.of(name(), blob)) - .contentType(MoreObjects.firstNonNull(contentType, Storage.DEFAULT_CONTENT_TYPE)).build(); - return storage.create(blobInfo, content, options); + BlobInfo blobInfo = BlobInfo.builder(BlobId.of(name(), blob)).contentType(contentType).build(); + StorageRpc.Tuple target = + BlobTargetOption.toTargetOptions(blobInfo, options); + return storage.create(target.x(), content, target.y()); } /** @@ -367,17 +651,54 @@ public Blob create(String blob, byte[] content, String contentType, BlobTargetOp * * @param blob a blob name * @param content the blob content as a stream - * @param contentType the blob content type. If {@code null} then - * {@value com.google.gcloud.storage.Storage#DEFAULT_CONTENT_TYPE} is used. + * @param contentType the blob content type * @param options options for blob creation * @return a complete blob information * @throws StorageException upon failure */ public Blob create(String blob, InputStream content, String contentType, BlobWriteOption... options) { - BlobInfo blobInfo = BlobInfo.builder(BlobId.of(name(), blob)) - .contentType(MoreObjects.firstNonNull(contentType, Storage.DEFAULT_CONTENT_TYPE)).build(); - return storage.create(blobInfo, content, options); + BlobInfo blobInfo = BlobInfo.builder(BlobId.of(name(), blob)).contentType(contentType).build(); + StorageRpc.Tuple write = + BlobWriteOption.toWriteOptions(blobInfo, options); + return storage.create(write.x(), content, write.y()); + } + + /** + * Creates a new blob in this bucket. Direct upload is used to upload {@code content}. + * For large content, {@link Blob#writer(com.google.gcloud.storage.Storage.BlobWriteOption...)} + * is recommended as it uses resumable upload. MD5 and CRC32C hashes of {@code content} are + * computed and used for validating transferred data. + * + * @param blob a blob name + * @param content the blob content + * @param options options for blob creation + * @return a complete blob information + * @throws StorageException upon failure + */ + public Blob create(String blob, byte[] content, BlobTargetOption... options) { + BlobInfo blobInfo = BlobInfo.builder(BlobId.of(name(), blob)).build(); + StorageRpc.Tuple target = + BlobTargetOption.toTargetOptions(blobInfo, options); + return storage.create(target.x(), content, target.y()); + } + + /** + * Creates a new blob in this bucket. Direct upload is used to upload {@code content}. + * For large content, {@link Blob#writer(com.google.gcloud.storage.Storage.BlobWriteOption...)} + * is recommended as it uses resumable upload. + * + * @param blob a blob name + * @param content the blob content as a stream + * @param options options for blob creation + * @return a complete blob information + * @throws StorageException upon failure + */ + public Blob create(String blob, InputStream content, BlobWriteOption... options) { + BlobInfo blobInfo = BlobInfo.builder(BlobId.of(name(), blob)).build(); + StorageRpc.Tuple write = + BlobWriteOption.toWriteOptions(blobInfo, options); + return storage.create(write.x(), content, write.y()); } /** diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BucketInfo.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BucketInfo.java index bf34413f417f..a1de1a07e03e 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BucketInfo.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/BucketInfo.java @@ -16,8 +16,8 @@ package com.google.gcloud.storage; -import static com.google.api.client.repackaged.com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.MoreObjects.firstNonNull; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.collect.Lists.transform; import com.google.api.client.json.jackson2.JacksonFactory; diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/CopyWriter.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/CopyWriter.java index 1e5427a847d4..743630b6c4c2 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/CopyWriter.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/CopyWriter.java @@ -22,9 +22,9 @@ import com.google.gcloud.Restorable; import com.google.gcloud.RestorableState; import com.google.gcloud.RetryHelper; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpc.RewriteRequest; -import com.google.gcloud.spi.StorageRpc.RewriteResponse; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc.RewriteRequest; +import com.google.gcloud.storage.spi.StorageRpc.RewriteResponse; import java.io.Serializable; import java.util.Map; @@ -32,7 +32,13 @@ import java.util.concurrent.Callable; /** - * Google Storage blob copy writer. This class holds the result of a copy request. If source and + * Google Storage blob copy writer. A {@code CopyWriter} object allows to copy both blob's data and + * information. To override source blob's information supply a {@code BlobInfo} to the + * {@code CopyRequest} using either + * {@link Storage.CopyRequest.Builder#target(BlobInfo, Storage.BlobTargetOption...)} or + * {@link Storage.CopyRequest.Builder#target(BlobInfo, Iterable)}. + * + *

This class holds the result of a copy request. If source and * destination blobs share the same location and storage class the copy is completed in one RPC call * otherwise one or more {@link #copyChunk} calls are necessary to complete the copy. In addition, * {@link CopyWriter#result()} can be used to automatically complete the copy and return information @@ -57,18 +63,19 @@ public class CopyWriter implements Restorable { * is {@code false} will block until all pending chunks are copied. * *

This method has the same effect of doing: - *

    {@code while (!copyWriter.isDone()) {
-   *        copyWriter.copyChunk();
-   *    }}
+   * 
 {@code
+   * while (!copyWriter.isDone()) {
+   *    copyWriter.copyChunk();
+   * }}
    * 
* * @throws StorageException upon failure */ - public BlobInfo result() { + public Blob result() { while (!isDone()) { copyChunk(); } - return BlobInfo.fromPb(rewriteResponse.result); + return Blob.fromPb(serviceOptions.service(), rewriteResponse.result); } /** @@ -119,8 +126,10 @@ public RestorableState capture() { serviceOptions, BlobId.fromPb(rewriteResponse.rewriteRequest.source), rewriteResponse.rewriteRequest.sourceOptions, + rewriteResponse.rewriteRequest.overrideInfo, BlobInfo.fromPb(rewriteResponse.rewriteRequest.target), rewriteResponse.rewriteRequest.targetOptions) + .result(rewriteResponse.result != null ? BlobInfo.fromPb(rewriteResponse.result) : null) .blobSize(blobSize()) .isDone(isDone()) .megabytesCopiedPerChunk(rewriteResponse.rewriteRequest.megabytesRewrittenPerCall) @@ -131,11 +140,12 @@ public RestorableState capture() { static class StateImpl implements RestorableState, Serializable { - private static final long serialVersionUID = 8279287678903181701L; + private static final long serialVersionUID = 1693964441435822700L; private final StorageOptions serviceOptions; private final BlobId source; private final Map sourceOptions; + private final boolean overrideInfo; private final BlobInfo target; private final Map targetOptions; private final BlobInfo result; @@ -149,6 +159,7 @@ static class StateImpl implements RestorableState, Serializable { this.serviceOptions = builder.serviceOptions; this.source = builder.source; this.sourceOptions = builder.sourceOptions; + this.overrideInfo = builder.overrideInfo; this.target = builder.target; this.targetOptions = builder.targetOptions; this.result = builder.result; @@ -164,6 +175,7 @@ static class Builder { private final StorageOptions serviceOptions; private final BlobId source; private final Map sourceOptions; + private final boolean overrideInfo; private final BlobInfo target; private final Map targetOptions; private BlobInfo result; @@ -174,11 +186,12 @@ static class Builder { private Long megabytesCopiedPerChunk; private Builder(StorageOptions options, BlobId source, - Map sourceOptions, - BlobInfo target, Map targetOptions) { + Map sourceOptions, boolean overrideInfo, BlobInfo target, + Map targetOptions) { this.serviceOptions = options; this.source = source; this.sourceOptions = sourceOptions; + this.overrideInfo = overrideInfo; this.target = target; this.targetOptions = targetOptions; } @@ -219,15 +232,15 @@ RestorableState build() { } static Builder builder(StorageOptions options, BlobId source, - Map sourceOptions, BlobInfo target, + Map sourceOptions, boolean overrideInfo, BlobInfo target, Map targetOptions) { - return new Builder(options, source, sourceOptions, target, targetOptions); + return new Builder(options, source, sourceOptions, overrideInfo, target, targetOptions); } @Override public CopyWriter restore() { - RewriteRequest rewriteRequest = new RewriteRequest( - source.toPb(), sourceOptions, target.toPb(), targetOptions, megabytesCopiedPerChunk); + RewriteRequest rewriteRequest = new RewriteRequest(source.toPb(), sourceOptions, + overrideInfo, target.toPb(), targetOptions, megabytesCopiedPerChunk); RewriteResponse rewriteResponse = new RewriteResponse(rewriteRequest, result != null ? result.toPb() : null, blobSize, isDone, rewriteToken, totalBytesCopied); @@ -236,8 +249,9 @@ public CopyWriter restore() { @Override public int hashCode() { - return Objects.hash(serviceOptions, source, sourceOptions, target, targetOptions, result, - blobSize, isDone, megabytesCopiedPerChunk, rewriteToken, totalBytesCopied); + return Objects.hash(serviceOptions, source, sourceOptions, overrideInfo, target, + targetOptions, result, blobSize, isDone, megabytesCopiedPerChunk, rewriteToken, + totalBytesCopied); } @Override @@ -252,6 +266,7 @@ public boolean equals(Object obj) { return Objects.equals(this.serviceOptions, other.serviceOptions) && Objects.equals(this.source, other.source) && Objects.equals(this.sourceOptions, other.sourceOptions) + && Objects.equals(this.overrideInfo, other.overrideInfo) && Objects.equals(this.target, other.target) && Objects.equals(this.targetOptions, other.targetOptions) && Objects.equals(this.result, other.result) @@ -266,10 +281,14 @@ public boolean equals(Object obj) { public String toString() { return MoreObjects.toStringHelper(this) .add("source", source) + .add("overrideInfo", overrideInfo) .add("target", target) - .add("isDone", isDone) - .add("totalBytesRewritten", totalBytesCopied) + .add("result", result) .add("blobSize", blobSize) + .add("isDone", isDone) + .add("rewriteToken", rewriteToken) + .add("totalBytesCopied", totalBytesCopied) + .add("megabytesCopiedPerChunk", megabytesCopiedPerChunk) .toString(); } } diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Option.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Option.java index 2ec8426bfa9f..65c55da7efc8 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Option.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Option.java @@ -19,7 +19,7 @@ import static com.google.common.base.Preconditions.checkNotNull; import com.google.common.base.MoreObjects; -import com.google.gcloud.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc; import java.io.Serializable; import java.util.Objects; diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Storage.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Storage.java index d1799daede3e..78f421e94e52 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Storage.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/Storage.java @@ -24,13 +24,14 @@ import com.google.common.collect.Iterables; import com.google.common.collect.Lists; import com.google.common.collect.Sets; +import com.google.gcloud.AuthCredentials; import com.google.gcloud.AuthCredentials.ServiceAccountAuthCredentials; import com.google.gcloud.Page; import com.google.gcloud.ReadChannel; import com.google.gcloud.Service; import com.google.gcloud.WriteChannel; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpc.Tuple; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc.Tuple; import java.io.InputStream; import java.io.Serializable; @@ -52,8 +53,6 @@ */ public interface Storage extends Service { - String DEFAULT_CONTENT_TYPE = "application/octet-stream"; - enum PredefinedAcl { AUTHENTICATED_READ("authenticatedRead"), ALL_AUTHENTICATED_USERS("allAuthenticatedUsers"), @@ -626,16 +625,16 @@ private BucketListOption(StorageRpc.Option option, Object value) { } /** - * Returns an option to specify the maximum number of buckets to be returned. + * Returns an option to specify the maximum number of buckets returned per page. */ - public static BucketListOption maxResults(long maxResults) { - return new BucketListOption(StorageRpc.Option.MAX_RESULTS, maxResults); + public static BucketListOption pageSize(long pageSize) { + return new BucketListOption(StorageRpc.Option.MAX_RESULTS, pageSize); } /** * Returns an option to specify the page token from which to start listing buckets. */ - public static BucketListOption startPageToken(String pageToken) { + public static BucketListOption pageToken(String pageToken) { return new BucketListOption(StorageRpc.Option.PAGE_TOKEN, pageToken); } @@ -655,7 +654,7 @@ public static BucketListOption prefix(String prefix) { */ public static BucketListOption fields(BucketField... fields) { StringBuilder builder = new StringBuilder(); - builder.append("items(").append(BucketField.selector(fields)).append(")"); + builder.append("items(").append(BucketField.selector(fields)).append("),nextPageToken"); return new BucketListOption(StorageRpc.Option.FIELDS, builder.toString()); } } @@ -672,16 +671,16 @@ private BlobListOption(StorageRpc.Option option, Object value) { } /** - * Returns an option to specify the maximum number of blobs to be returned. + * Returns an option to specify the maximum number of blobs returned per page. */ - public static BlobListOption maxResults(long maxResults) { - return new BlobListOption(StorageRpc.Option.MAX_RESULTS, maxResults); + public static BlobListOption pageSize(long pageSize) { + return new BlobListOption(StorageRpc.Option.MAX_RESULTS, pageSize); } /** * Returns an option to specify the page token from which to start listing blobs. */ - public static BlobListOption startPageToken(String pageToken) { + public static BlobListOption pageToken(String pageToken) { return new BlobListOption(StorageRpc.Option.PAGE_TOKEN, pageToken); } @@ -694,10 +693,26 @@ public static BlobListOption prefix(String prefix) { } /** - * Returns an option to specify whether blob listing should include subdirectories or not. + * If specified, results are returned in a directory-like mode. Blobs whose names, after a + * possible {@link #prefix(String)}, do not contain the '/' delimiter are returned as is. Blobs + * whose names, after a possible {@link #prefix(String)}, contain the '/' delimiter, will have + * their name truncated after the delimiter and will be returned as {@link Blob} objects where + * only {@link Blob#blobId()}, {@link Blob#size()} and {@link Blob#isDirectory()} are set. For + * such directory blobs, ({@link BlobId#generation()} returns {@code null}), {@link Blob#size()} + * returns {@code 0} while {@link Blob#isDirectory()} returns {@code true}. Duplicate directory + * blobs are omitted. + */ + public static BlobListOption currentDirectory() { + return new BlobListOption(StorageRpc.Option.DELIMITER, true); + } + + /** + * If set to {@code true}, lists all versions of a blob. The default is {@code false}. + * + * @see Object Versioning */ - public static BlobListOption recursive(boolean recursive) { - return new BlobListOption(StorageRpc.Option.DELIMITER, recursive); + public static BlobListOption versions(boolean versions) { + return new BlobListOption(StorageRpc.Option.VERSIONS, versions); } /** @@ -708,7 +723,7 @@ public static BlobListOption recursive(boolean recursive) { */ public static BlobListOption fields(BlobField... fields) { StringBuilder builder = new StringBuilder(); - builder.append("items(").append(BlobField.selector(fields)).append(")"); + builder.append("items(").append(BlobField.selector(fields)).append("),nextPageToken"); return new BlobListOption(StorageRpc.Option.FIELDS, builder.toString()); } } @@ -947,6 +962,7 @@ class CopyRequest implements Serializable { private final BlobId source; private final List sourceOptions; + private final boolean overrideInfo; private final BlobInfo target; private final List targetOptions; private final Long megabytesCopiedPerChunk; @@ -956,6 +972,7 @@ public static class Builder { private final Set sourceOptions = new LinkedHashSet<>(); private final Set targetOptions = new LinkedHashSet<>(); private BlobId source; + private boolean overrideInfo; private BlobInfo target; private Long megabytesCopiedPerChunk; @@ -1004,39 +1021,38 @@ public Builder sourceOptions(Iterable options) { * * @return the builder */ - public Builder target(BlobId target) { - this.target = BlobInfo.builder(target).build(); + public Builder target(BlobId targetId) { + this.overrideInfo = false; + this.target = BlobInfo.builder(targetId).build(); return this; } /** * Sets the copy target and target options. {@code target} parameter is used to override - * source blob information (e.g. {@code contentType}, {@code contentLanguage}). {@code - * target.contentType} is a required field. + * source blob information (e.g. {@code contentType}, {@code contentLanguage}). Target blob + * information is set exactly to {@code target}, no information is inherited from the source + * blob. * * @return the builder - * @throws IllegalArgumentException if {@code target.contentType} is {@code null} */ - public Builder target(BlobInfo target, BlobTargetOption... options) - throws IllegalArgumentException { - checkContentType(target); - this.target = target; + public Builder target(BlobInfo target, BlobTargetOption... options) { + this.overrideInfo = true; + this.target = checkNotNull(target); Collections.addAll(targetOptions, options); return this; } /** * Sets the copy target and target options. {@code target} parameter is used to override - * source blob information (e.g. {@code contentType}, {@code contentLanguage}). {@code - * target.contentType} is a required field. + * source blob information (e.g. {@code contentType}, {@code contentLanguage}). Target blob + * information is set exactly to {@code target}, no information is inherited from the source + * blob. * * @return the builder - * @throws IllegalArgumentException if {@code target.contentType} is {@code null} */ - public Builder target(BlobInfo target, Iterable options) - throws IllegalArgumentException { - checkContentType(target); - this.target = target; + public Builder target(BlobInfo target, Iterable options) { + this.overrideInfo = true; + this.target = checkNotNull(target); Iterables.addAll(targetOptions, options); return this; } @@ -1057,8 +1073,6 @@ public Builder megabytesCopiedPerChunk(Long megabytesCopiedPerChunk) { * Creates a {@code CopyRequest} object. */ public CopyRequest build() { - checkNotNull(source); - checkNotNull(target); return new CopyRequest(this); } } @@ -1066,6 +1080,7 @@ public CopyRequest build() { private CopyRequest(Builder builder) { source = checkNotNull(builder.source); sourceOptions = ImmutableList.copyOf(builder.sourceOptions); + overrideInfo = builder.overrideInfo; target = checkNotNull(builder.target); targetOptions = ImmutableList.copyOf(builder.targetOptions); megabytesCopiedPerChunk = builder.megabytesCopiedPerChunk; @@ -1092,6 +1107,17 @@ public BlobInfo target() { return target; } + /** + * Returns whether to override the target blob information with {@link #target()}. + * If {@code true}, the value of {@link #target()} is used to replace source blob information + * (e.g. {@code contentType}, {@code contentLanguage}). Target blob information is set exactly + * to this value, no information is inherited from the source blob. If {@code false}, target + * blob information is inherited from the source blob. + */ + public boolean overrideInfo() { + return overrideInfo; + } + /** * Returns blob's target options. */ @@ -1110,34 +1136,27 @@ public Long megabytesCopiedPerChunk() { /** * Creates a copy request. {@code target} parameter is used to override source blob information - * (e.g. {@code contentType}, {@code contentLanguage}). {@code target.contentType} is a required - * field. + * (e.g. {@code contentType}, {@code contentLanguage}). * * @param sourceBucket name of the bucket containing the source blob * @param sourceBlob name of the source blob * @param target a {@code BlobInfo} object for the target blob * @return a copy request - * @throws IllegalArgumentException if {@code target.contentType} is {@code null} */ - public static CopyRequest of(String sourceBucket, String sourceBlob, BlobInfo target) - throws IllegalArgumentException { - checkContentType(target); + public static CopyRequest of(String sourceBucket, String sourceBlob, BlobInfo target) { return builder().source(sourceBucket, sourceBlob).target(target).build(); } /** - * Creates a copy request. {@code target} parameter is used to override source blob information - * (e.g. {@code contentType}, {@code contentLanguage}). {@code target.contentType} is a required - * field. + * Creates a copy request. {@code target} parameter is used to replace source blob information + * (e.g. {@code contentType}, {@code contentLanguage}). Target blob information is set exactly + * to {@code target}, no information is inherited from the source blob. * * @param sourceBlobId a {@code BlobId} object for the source blob * @param target a {@code BlobInfo} object for the target blob * @return a copy request - * @throws IllegalArgumentException if {@code target.contentType} is {@code null} */ - public static CopyRequest of(BlobId sourceBlobId, BlobInfo target) - throws IllegalArgumentException { - checkContentType(target); + public static CopyRequest of(BlobId sourceBlobId, BlobInfo target) { return builder().source(sourceBlobId).target(target).build(); } @@ -1199,14 +1218,10 @@ public static CopyRequest of(BlobId sourceBlobId, BlobId targetBlobId) { public static Builder builder() { return new Builder(); } - - private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentException { - checkArgument(blobInfo.contentType() != null, "Blob content type can not be null"); - } } /** - * Create a new bucket. + * Creates a new bucket. * * @return a complete bucket * @throws StorageException upon failure @@ -1214,7 +1229,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx Bucket create(BucketInfo bucketInfo, BucketTargetOption... options); /** - * Create a new blob with no content. + * Creates a new blob with no content. * * @return a [@code Blob} with complete information * @throws StorageException upon failure @@ -1222,7 +1237,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx Blob create(BlobInfo blobInfo, BlobTargetOption... options); /** - * Create a new blob. Direct upload is used to upload {@code content}. For large content, + * Creates a new blob. Direct upload is used to upload {@code content}. For large content, * {@link #writer} is recommended as it uses resumable upload. MD5 and CRC32C hashes of * {@code content} are computed and used for validating transferred data. * @@ -1233,7 +1248,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx Blob create(BlobInfo blobInfo, byte[] content, BlobTargetOption... options); /** - * Create a new blob. Direct upload is used to upload {@code content}. For large content, + * Creates a new blob. Direct upload is used to upload {@code content}. For large content, * {@link #writer} is recommended as it uses resumable upload. By default any md5 and crc32c * values in the given {@code blobInfo} are ignored unless requested via the * {@code BlobWriteOption.md5Match} and {@code BlobWriteOption.crc32cMatch} options. The given @@ -1245,49 +1260,50 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx Blob create(BlobInfo blobInfo, InputStream content, BlobWriteOption... options); /** - * Return the requested bucket or {@code null} if not found. + * Returns the requested bucket or {@code null} if not found. * * @throws StorageException upon failure */ Bucket get(String bucket, BucketGetOption... options); /** - * Return the requested blob or {@code null} if not found. + * Returns the requested blob or {@code null} if not found. * * @throws StorageException upon failure */ Blob get(String bucket, String blob, BlobGetOption... options); /** - * Return the requested blob or {@code null} if not found. + * Returns the requested blob or {@code null} if not found. * * @throws StorageException upon failure */ Blob get(BlobId blob, BlobGetOption... options); /** - * Return the requested blob or {@code null} if not found. + * Returns the requested blob or {@code null} if not found. * * @throws StorageException upon failure */ Blob get(BlobId blob); /** - * List the project's buckets. + * Lists the project's buckets. * * @throws StorageException upon failure */ Page list(BucketListOption... options); /** - * List the bucket's blobs. + * Lists the bucket's blobs. If the {@link BlobListOption#currentDirectory()} option is provided, + * results are returned in a directory-like mode. * * @throws StorageException upon failure */ Page list(String bucket, BlobListOption... options); /** - * Update bucket information. + * Updates bucket information. * * @return the updated bucket * @throws StorageException upon failure @@ -1295,13 +1311,15 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx Bucket update(BucketInfo bucketInfo, BucketTargetOption... options); /** - * Update blob information. Original metadata are merged with metadata in the provided + * Updates blob information. Original metadata are merged with metadata in the provided * {@code blobInfo}. To replace metadata instead you first have to unset them. Unsetting metadata * can be done by setting the provided {@code blobInfo}'s metadata to {@code null}. * *

Example usage of replacing blob's metadata: - *

    {@code service.update(BlobInfo.builder("bucket", "name").metadata(null).build());}
-   *    {@code service.update(BlobInfo.builder("bucket", "name").metadata(newMetadata).build());}
+   * 
 {@code
+   * service.update(BlobInfo.builder("bucket", "name").metadata(null).build());
+   * service.update(BlobInfo.builder("bucket", "name").metadata(newMetadata).build());
+   * }
    * 
* * @return the updated blob @@ -1310,13 +1328,15 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx Blob update(BlobInfo blobInfo, BlobTargetOption... options); /** - * Update blob information. Original metadata are merged with metadata in the provided + * Updates blob information. Original metadata are merged with metadata in the provided * {@code blobInfo}. To replace metadata instead you first have to unset them. Unsetting metadata * can be done by setting the provided {@code blobInfo}'s metadata to {@code null}. * *

Example usage of replacing blob's metadata: - *

    {@code service.update(BlobInfo.builder("bucket", "name").metadata(null).build());}
-   *    {@code service.update(BlobInfo.builder("bucket", "name").metadata(newMetadata).build());}
+   * 
 {@code
+   * service.update(BlobInfo.builder("bucket", "name").metadata(null).build());
+   * service.update(BlobInfo.builder("bucket", "name").metadata(newMetadata).build());
+   * }
    * 
* * @return the updated blob @@ -1325,7 +1345,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx Blob update(BlobInfo blobInfo); /** - * Delete the requested bucket. + * Deletes the requested bucket. * * @return {@code true} if bucket was deleted, {@code false} if it was not found * @throws StorageException upon failure @@ -1333,7 +1353,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx boolean delete(String bucket, BucketSourceOption... options); /** - * Delete the requested blob. + * Deletes the requested blob. * * @return {@code true} if blob was deleted, {@code false} if it was not found * @throws StorageException upon failure @@ -1341,7 +1361,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx boolean delete(String bucket, String blob, BlobSourceOption... options); /** - * Delete the requested blob. + * Deletes the requested blob. * * @return {@code true} if blob was deleted, {@code false} if it was not found * @throws StorageException upon failure @@ -1349,7 +1369,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx boolean delete(BlobId blob, BlobSourceOption... options); /** - * Delete the requested blob. + * Deletes the requested blob. * * @return {@code true} if blob was deleted, {@code false} if it was not found * @throws StorageException upon failure @@ -1357,7 +1377,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx boolean delete(BlobId blob); /** - * Send a compose request. + * Sends a compose request. * * @return the composed blob * @throws StorageException upon failure @@ -1365,22 +1385,29 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx Blob compose(ComposeRequest composeRequest); /** - * Sends a copy request. Returns a {@link CopyWriter} object for the provided - * {@code CopyRequest}. If source and destination objects share the same location and storage - * class the source blob is copied with one request and {@link CopyWriter#result()} immediately - * returns, regardless of the {@link CopyRequest#megabytesCopiedPerChunk} parameter. - * If source and destination have different location or storage class {@link CopyWriter#result()} - * might issue multiple RPC calls depending on blob's size. + * Sends a copy request. This method copies both blob's data and information. To override source + * blob's information supply a {@code BlobInfo} to the + * {@code CopyRequest} using either + * {@link Storage.CopyRequest.Builder#target(BlobInfo, Storage.BlobTargetOption...)} or + * {@link Storage.CopyRequest.Builder#target(BlobInfo, Iterable)}. + * + *

This method returns a {@link CopyWriter} object for the provided {@code CopyRequest}. If + * source and destination objects share the same location and storage class the source blob is + * copied with one request and {@link CopyWriter#result()} immediately returns, regardless of the + * {@link CopyRequest#megabytesCopiedPerChunk} parameter. If source and destination have different + * location or storage class {@link CopyWriter#result()} might issue multiple RPC calls depending + * on blob's size. * *

Example usage of copy: - *

    {@code BlobInfo blob = service.copy(copyRequest).result();}
+   * 
 {@code BlobInfo blob = service.copy(copyRequest).result();}
    * 
* To explicitly issue chunk copy requests use {@link CopyWriter#copyChunk()} instead: - *
    {@code CopyWriter copyWriter = service.copy(copyRequest);
-   *    while (!copyWriter.isDone()) {
-   *        copyWriter.copyChunk();
-   *    }
-   *    BlobInfo blob = copyWriter.result();
+   * 
 {@code
+   * CopyWriter copyWriter = service.copy(copyRequest);
+   * while (!copyWriter.isDone()) {
+   *     copyWriter.copyChunk();
+   * }
+   * BlobInfo blob = copyWriter.result();
    * }
    * 
* @@ -1408,7 +1435,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx byte[] readAllBytes(BlobId blob, BlobSourceOption... options); /** - * Send a batch request. + * Sends a batch request. * * @return the batch response * @throws StorageException upon failure @@ -1416,7 +1443,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx BatchResponse submit(BatchRequest batchRequest); /** - * Return a channel for reading the blob's content. The blob's latest generation is read. If the + * Returns a channel for reading the blob's content. The blob's latest generation is read. If the * blob changes while reading (i.e. {@link BlobInfo#etag()} changes), subsequent calls to * {@code blobReadChannel.read(ByteBuffer)} may throw {@link StorageException}. * @@ -1429,7 +1456,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx ReadChannel reader(String bucket, String blob, BlobSourceOption... options); /** - * Return a channel for reading the blob's content. If {@code blob.generation()} is set + * Returns a channel for reading the blob's content. If {@code blob.generation()} is set * data corresponding to that generation is read. If {@code blob.generation()} is {@code null} * the blob's latest generation is read. If the blob changes while reading (i.e. * {@link BlobInfo#etag()} changes), subsequent calls to {@code blobReadChannel.read(ByteBuffer)} @@ -1445,7 +1472,7 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx ReadChannel reader(BlobId blob, BlobSourceOption... options); /** - * Create a blob and return a channel for writing its content. By default any md5 and crc32c + * Creates a blob and return a channel for writing its content. By default any md5 and crc32c * values in the given {@code blobInfo} are ignored unless requested via the * {@code BlobWriteOption.md5Match} and {@code BlobWriteOption.crc32cMatch} options. * @@ -1454,16 +1481,33 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx WriteChannel writer(BlobInfo blobInfo, BlobWriteOption... options); /** - * Generates a signed URL for a blob. - * If you have a blob that you want to allow access to for a fixed - * amount of time, you can use this method to generate a URL that - * is only valid within a certain time period. - * This is particularly useful if you don't want publicly - * accessible blobs, but don't want to require users to explicitly log in. + * Generates a signed URL for a blob. If you have a blob that you want to allow access to for a + * fixed amount of time, you can use this method to generate a URL that is only valid within a + * certain time period. This is particularly useful if you don't want publicly accessible blobs, + * but also don't want to require users to explicitly log in. Signing a URL requires a service + * account and its associated private key. If a {@link ServiceAccountAuthCredentials} was passed + * to {@link StorageOptions.Builder#authCredentials(AuthCredentials)} or the default credentials + * are being used and the environment variable {@code GOOGLE_APPLICATION_CREDENTIALS} is set, then + * {@code signUrl} will use that service account and associated key to sign the URL. If the + * credentials passed to {@link StorageOptions} do not expose a private key (this is the case for + * App Engine credentials, Compute Engine credentials and Google Cloud SDK credentials) then + * {@code signUrl} will throw an {@link IllegalArgumentException} unless a service account with + * associated key is passed using the {@code SignUrlOption.serviceAccount()} option. The service + * account and private key passed with {@code SignUrlOption.serviceAccount()} have priority over + * any credentials set with {@link StorageOptions.Builder#authCredentials(AuthCredentials)}. + * + *

Example usage of creating a signed URL that is valid for 2 weeks, using the default + * credentials for signing the URL: + *

 {@code
+   * service.signUrl(BlobInfo.builder("bucket", "name").build(), 14, TimeUnit.DAYS);
+   * }
* - *

Example usage of creating a signed URL that is valid for 2 weeks: - *

   {@code
-   *     service.signUrl(BlobInfo.builder("bucket", "name").build(), 14, TimeUnit.DAYS);
+   * 

Example usage of creating a signed URL passing the {@code SignUrlOption.serviceAccount()} + * option, that will be used for signing the URL: + *

 {@code
+   * service.signUrl(BlobInfo.builder("bucket", "name").build(), 14, TimeUnit.DAYS,
+   *     SignUrlOption.serviceAccount(
+   *         AuthCredentials.createForJson(new FileInputStream("/path/to/key.json"))));
    * }
* * @param blobInfo the blob associated with the signed URL @@ -1471,6 +1515,14 @@ private static void checkContentType(BlobInfo blobInfo) throws IllegalArgumentEx * granularity supported is 1 second, finer granularities will be truncated * @param unit time unit of the {@code duration} parameter * @param options optional URL signing options + * @throws IllegalArgumentException if {@code SignUrlOption.serviceAccount()} was not used and no + * service account was provided to {@link StorageOptions} + * @throws IllegalArgumentException if the key associated to the provided service account is + * invalid + * @throws IllegalArgumentException if {@code SignUrlOption.withMd5()} option is used and + * {@code blobInfo.md5()} is {@code null} + * @throws IllegalArgumentException if {@code SignUrlOption.withContentType()} option is used and + * {@code blobInfo.contentType()} is {@code null} * @see Signed-URLs */ URL signUrl(BlobInfo blobInfo, long duration, TimeUnit unit, SignUrlOption... options); diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageException.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageException.java index 0c952c9a65d6..ee85b80d6e13 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageException.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageException.java @@ -33,7 +33,7 @@ */ public class StorageException extends BaseServiceException { - // see: https://cloud.google.com/storage/docs/concepts-techniques#practices + // see: https://cloud.google.com/storage/docs/resumable-uploads-xml#practices private static final Set RETRYABLE_ERRORS = ImmutableSet.of( new Error(504, null), new Error(503, null), diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageImpl.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageImpl.java index de77cba021a1..cf709ba5e293 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageImpl.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageImpl.java @@ -19,15 +19,15 @@ import static com.google.common.base.MoreObjects.firstNonNull; import static com.google.common.base.Preconditions.checkArgument; import static com.google.gcloud.RetryHelper.runWithRetries; -import static com.google.gcloud.spi.StorageRpc.Option.DELIMITER; -import static com.google.gcloud.spi.StorageRpc.Option.IF_GENERATION_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_GENERATION_NOT_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_METAGENERATION_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_METAGENERATION_NOT_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_SOURCE_GENERATION_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_SOURCE_GENERATION_NOT_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_SOURCE_METAGENERATION_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_SOURCE_METAGENERATION_NOT_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.DELIMITER; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_GENERATION_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_GENERATION_NOT_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_METAGENERATION_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_METAGENERATION_NOT_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_SOURCE_GENERATION_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_SOURCE_GENERATION_NOT_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_SOURCE_METAGENERATION_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_SOURCE_METAGENERATION_NOT_MATCH; import static java.nio.charset.StandardCharsets.UTF_8; import com.google.api.services.storage.model.StorageObject; @@ -48,9 +48,9 @@ import com.google.gcloud.PageImpl.NextPageFetcher; import com.google.gcloud.ReadChannel; import com.google.gcloud.RetryHelper.RetryHelperException; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpc.RewriteResponse; -import com.google.gcloud.spi.StorageRpc.Tuple; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc.RewriteResponse; +import com.google.gcloud.storage.spi.StorageRpc.Tuple; import java.io.ByteArrayInputStream; import java.io.InputStream; @@ -76,6 +76,7 @@ final class StorageImpl extends BaseService implements Storage { private static final byte[] EMPTY_BYTE_ARRAY = {}; private static final String EMPTY_BYTE_ARRAY_MD5 = "1B2M2Y8AsgTpgAmY7PhCfg=="; private static final String EMPTY_BYTE_ARRAY_CRC32C = "AAAAAA=="; + private static final String PATH_DELIMITER = "/"; private static final Function, Boolean> DELETE_FUNCTION = new Function, Boolean>() { @@ -412,15 +413,16 @@ public CopyWriter copy(final CopyRequest copyRequest) { final StorageObject source = copyRequest.source().toPb(); final Map sourceOptions = optionMap(copyRequest.source().generation(), null, copyRequest.sourceOptions(), true); - final StorageObject target = copyRequest.target().toPb(); + final StorageObject targetObject = copyRequest.target().toPb(); final Map targetOptions = optionMap(copyRequest.target().generation(), copyRequest.target().metageneration(), copyRequest.targetOptions()); try { RewriteResponse rewriteResponse = runWithRetries(new Callable() { @Override public RewriteResponse call() { - return storageRpc.openRewrite(new StorageRpc.RewriteRequest(source, sourceOptions, target, - targetOptions, copyRequest.megabytesCopiedPerChunk())); + return storageRpc.openRewrite(new StorageRpc.RewriteRequest(source, sourceOptions, + copyRequest.overrideInfo(), targetObject, targetOptions, + copyRequest.megabytesCopiedPerChunk())); } }, options().retryParams(), EXCEPTION_HANDLER); return new CopyWriter(options(), rewriteResponse); @@ -669,7 +671,7 @@ private static void addToOptionMap(StorageRpc.Option getOption, StorageRpc.O } Boolean value = (Boolean) temp.remove(DELIMITER); if (Boolean.TRUE.equals(value)) { - temp.put(DELIMITER, options().pathDelimiter()); + temp.put(DELIMITER, PATH_DELIMITER); } if (useAsSource) { addToOptionMap(IF_GENERATION_MATCH, IF_SOURCE_GENERATION_MATCH, generation, temp); diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageOptions.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageOptions.java index bd30cb173366..e7e1c2778fa9 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageOptions.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/StorageOptions.java @@ -16,14 +16,12 @@ package com.google.gcloud.storage; -import com.google.common.base.MoreObjects; import com.google.common.collect.ImmutableSet; import com.google.gcloud.ServiceOptions; -import com.google.gcloud.spi.DefaultStorageRpc; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpcFactory; +import com.google.gcloud.storage.spi.DefaultStorageRpc; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpcFactory; -import java.util.Objects; import java.util.Set; public class StorageOptions extends ServiceOptions { @@ -31,9 +29,6 @@ public class StorageOptions extends ServiceOptions SCOPES = ImmutableSet.of(GCS_SCOPE); - private static final String DEFAULT_PATH_DELIMITER = "/"; - - private final String pathDelimiter; public static class DefaultStorageFactory implements StorageFactory { @@ -58,24 +53,10 @@ public StorageRpc create(StorageOptions options) { public static class Builder extends ServiceOptions.Builder { - private String pathDelimiter; - private Builder() {} private Builder(StorageOptions options) { super(options); - pathDelimiter = options.pathDelimiter; - } - - /** - * Sets the path delimiter for the storage service. - * - * @param pathDelimiter the path delimiter to set - * @return the builder - */ - public Builder pathDelimiter(String pathDelimiter) { - this.pathDelimiter = pathDelimiter; - return this; } @Override @@ -86,7 +67,6 @@ public StorageOptions build() { private StorageOptions(Builder builder) { super(StorageFactory.class, StorageRpcFactory.class, builder); - pathDelimiter = MoreObjects.firstNonNull(builder.pathDelimiter, DEFAULT_PATH_DELIMITER); } @SuppressWarnings("unchecked") @@ -106,13 +86,6 @@ protected Set scopes() { return SCOPES; } - /** - * Returns the storage service's path delimiter. - */ - public String pathDelimiter() { - return pathDelimiter; - } - /** * Returns a default {@code StorageOptions} instance. */ @@ -128,16 +101,12 @@ public Builder toBuilder() { @Override public int hashCode() { - return baseHashCode() ^ Objects.hash(pathDelimiter); + return baseHashCode(); } @Override public boolean equals(Object obj) { - if (!(obj instanceof StorageOptions)) { - return false; - } - StorageOptions other = (StorageOptions) obj; - return baseEquals(other) && Objects.equals(pathDelimiter, other.pathDelimiter); + return obj instanceof StorageOptions && baseEquals((StorageOptions) obj); } public static Builder builder() { diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/spi/DefaultStorageRpc.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/DefaultStorageRpc.java similarity index 86% rename from gcloud-java-storage/src/main/java/com/google/gcloud/spi/DefaultStorageRpc.java rename to gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/DefaultStorageRpc.java index dc84a1de5559..8d06832534e2 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/spi/DefaultStorageRpc.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/DefaultStorageRpc.java @@ -12,25 +12,27 @@ * the License. */ -package com.google.gcloud.spi; - -import static com.google.gcloud.spi.StorageRpc.Option.DELIMITER; -import static com.google.gcloud.spi.StorageRpc.Option.FIELDS; -import static com.google.gcloud.spi.StorageRpc.Option.IF_GENERATION_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_GENERATION_NOT_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_METAGENERATION_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_METAGENERATION_NOT_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_SOURCE_GENERATION_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_SOURCE_GENERATION_NOT_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_SOURCE_METAGENERATION_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.IF_SOURCE_METAGENERATION_NOT_MATCH; -import static com.google.gcloud.spi.StorageRpc.Option.MAX_RESULTS; -import static com.google.gcloud.spi.StorageRpc.Option.PAGE_TOKEN; -import static com.google.gcloud.spi.StorageRpc.Option.PREDEFINED_ACL; -import static com.google.gcloud.spi.StorageRpc.Option.PREDEFINED_DEFAULT_OBJECT_ACL; -import static com.google.gcloud.spi.StorageRpc.Option.PREFIX; -import static com.google.gcloud.spi.StorageRpc.Option.VERSIONS; +package com.google.gcloud.storage.spi; + +import static com.google.common.base.MoreObjects.firstNonNull; +import static com.google.gcloud.storage.spi.StorageRpc.Option.DELIMITER; +import static com.google.gcloud.storage.spi.StorageRpc.Option.FIELDS; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_GENERATION_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_GENERATION_NOT_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_METAGENERATION_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_METAGENERATION_NOT_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_SOURCE_GENERATION_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_SOURCE_GENERATION_NOT_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_SOURCE_METAGENERATION_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.IF_SOURCE_METAGENERATION_NOT_MATCH; +import static com.google.gcloud.storage.spi.StorageRpc.Option.MAX_RESULTS; +import static com.google.gcloud.storage.spi.StorageRpc.Option.PAGE_TOKEN; +import static com.google.gcloud.storage.spi.StorageRpc.Option.PREDEFINED_ACL; +import static com.google.gcloud.storage.spi.StorageRpc.Option.PREDEFINED_DEFAULT_OBJECT_ACL; +import static com.google.gcloud.storage.spi.StorageRpc.Option.PREFIX; +import static com.google.gcloud.storage.spi.StorageRpc.Option.VERSIONS; import static java.net.HttpURLConnection.HTTP_NOT_FOUND; +import static javax.servlet.http.HttpServletResponse.SC_REQUESTED_RANGE_NOT_SATISFIABLE; import com.google.api.client.googleapis.batch.json.JsonBatchCallback; import com.google.api.client.googleapis.json.GoogleJsonError; @@ -56,8 +58,9 @@ import com.google.api.services.storage.model.ComposeRequest.SourceObjects.ObjectPreconditions; import com.google.api.services.storage.model.Objects; import com.google.api.services.storage.model.StorageObject; -import com.google.common.base.MoreObjects; +import com.google.common.base.Function; import com.google.common.collect.ImmutableList; +import com.google.common.collect.Iterables; import com.google.common.collect.Lists; import com.google.common.collect.Maps; import com.google.gcloud.storage.StorageException; @@ -66,6 +69,7 @@ import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.InputStream; +import java.math.BigInteger; import java.util.ArrayList; import java.util.Iterator; import java.util.List; @@ -99,7 +103,7 @@ private static StorageException translate(GoogleJsonError exception) { } @Override - public Bucket create(Bucket bucket, Map options) throws StorageException { + public Bucket create(Bucket bucket, Map options) { try { return storage.buckets() .insert(this.options.projectId(), bucket) @@ -114,7 +118,7 @@ public Bucket create(Bucket bucket, Map options) throws StorageExcept @Override public StorageObject create(StorageObject storageObject, final InputStream content, - Map options) throws StorageException { + Map options) { try { Storage.Objects.Insert insert = storage.objects() .insert(storageObject.getBucket(), storageObject, @@ -150,7 +154,7 @@ public Tuple> list(Map options) { } @Override - public Tuple> list(String bucket, Map options) { + public Tuple> list(final String bucket, Map options) { try { Objects objects = storage.objects() .list(bucket) @@ -162,13 +166,30 @@ public Tuple> list(String bucket, Map .setPageToken(PAGE_TOKEN.getString(options)) .setFields(FIELDS.getString(options)) .execute(); - return Tuple.>of( - objects.getNextPageToken(), objects.getItems()); + Iterable storageObjects = Iterables.concat( + firstNonNull(objects.getItems(), ImmutableList.of()), + objects.getPrefixes() != null + ? Lists.transform(objects.getPrefixes(), objectFromPrefix(bucket)) + : ImmutableList.of()); + return Tuple.of(objects.getNextPageToken(), storageObjects); } catch (IOException ex) { throw translate(ex); } } + private static Function objectFromPrefix(final String bucket) { + return new Function() { + @Override + public StorageObject apply(String prefix) { + return new StorageObject() + .set("isDirectory", true) + .setBucket(bucket) + .setName(prefix) + .setSize(BigInteger.ZERO); + } + }; + } + @Override public Bucket get(Bucket bucket, Map options) { try { @@ -296,11 +317,8 @@ private Storage.Objects.Delete deleteRequest(StorageObject blob, Map @Override public StorageObject compose(Iterable sources, StorageObject target, - Map targetOptions) throws StorageException { + Map targetOptions) { ComposeRequest request = new ComposeRequest(); - if (target.getContentType() == null) { - target.setContentType("application/octet-stream"); - } request.setDestination(target); List sourceObjects = new ArrayList<>(); for (StorageObject source : sources) { @@ -327,8 +345,7 @@ public StorageObject compose(Iterable sources, StorageObject targ } @Override - public byte[] load(StorageObject from, Map options) - throws StorageException { + public byte[] load(StorageObject from, Map options) { try { Storage.Objects.Get getRequest = storage.objects() .get(from.getBucket(), from.getName()) @@ -347,7 +364,7 @@ public byte[] load(StorageObject from, Map options) } @Override - public BatchResponse batch(BatchRequest request) throws StorageException { + public BatchResponse batch(BatchRequest request) { List>>> partitionedToDelete = Lists.partition(request.toDelete, MAX_BATCH_DELETES); Iterator>>> iterator = partitionedToDelete.iterator(); @@ -437,7 +454,7 @@ public void onFailure(GoogleJsonError e, HttpHeaders responseHeaders) { @Override public Tuple read(StorageObject from, Map options, long position, - int bytes) throws StorageException { + int bytes) { try { Get req = storage.objects() .get(from.getBucket(), from.getName()) @@ -454,20 +471,32 @@ public Tuple read(StorageObject from, Map options, lo String etag = req.getLastResponseHeaders().getETag(); return Tuple.of(etag, output.toByteArray()); } catch (IOException ex) { - throw translate(ex); + StorageException serviceException = translate(ex); + if (serviceException.code() == SC_REQUESTED_RANGE_NOT_SATISFIABLE) { + return Tuple.of(null, new byte[0]); + } + throw serviceException; } } @Override public void write(String uploadId, byte[] toWrite, int toWriteOffset, long destOffset, int length, - boolean last) throws StorageException { + boolean last) { try { + if (length == 0 && !last) { + return; + } GenericUrl url = new GenericUrl(uploadId); HttpRequest httpRequest = storage.getRequestFactory().buildPutRequest(url, new ByteArrayContent(null, toWrite, toWriteOffset, length)); long limit = destOffset + length; StringBuilder range = new StringBuilder("bytes "); - range.append(destOffset).append('-').append(limit - 1).append('/'); + if (length == 0) { + range.append('*'); + } else { + range.append(destOffset).append('-').append(limit - 1); + } + range.append('/'); if (last) { range.append(limit); } else { @@ -501,8 +530,7 @@ public void write(String uploadId, byte[] toWrite, int toWriteOffset, long destO } @Override - public String open(StorageObject object, Map options) - throws StorageException { + public String open(StorageObject object, Map options) { try { Insert req = storage.objects().insert(object.getBucket(), object); GenericUrl url = req.buildHttpRequest().getUrl(); @@ -523,7 +551,7 @@ public String open(StorageObject object, Map options) HttpRequest httpRequest = requestFactory.buildPostRequest(url, new JsonHttpContent(jsonFactory, object)); httpRequest.getHeaders().set("X-Upload-Content-Type", - MoreObjects.firstNonNull(object.getContentType(), "application/octet-stream")); + firstNonNull(object.getContentType(), "application/octet-stream")); HttpResponse response = httpRequest.execute(); if (response.getStatusCode() != 200) { GoogleJsonError error = new GoogleJsonError(); @@ -538,22 +566,22 @@ public String open(StorageObject object, Map options) } @Override - public RewriteResponse openRewrite(RewriteRequest rewriteRequest) throws StorageException { + public RewriteResponse openRewrite(RewriteRequest rewriteRequest) { return rewrite(rewriteRequest, null); } @Override - public RewriteResponse continueRewrite(RewriteResponse previousResponse) throws StorageException { + public RewriteResponse continueRewrite(RewriteResponse previousResponse) { return rewrite(previousResponse.rewriteRequest, previousResponse.rewriteToken); } - private RewriteResponse rewrite(RewriteRequest req, String token) throws StorageException { + private RewriteResponse rewrite(RewriteRequest req, String token) { try { Long maxBytesRewrittenPerCall = req.megabytesRewrittenPerCall != null ? req.megabytesRewrittenPerCall * MEGABYTE : null; com.google.api.services.storage.model.RewriteResponse rewriteResponse = storage.objects() .rewrite(req.source.getBucket(), req.source.getName(), req.target.getBucket(), - req.target.getName(), req.target.getContentType() != null ? req.target : null) + req.target.getName(), req.overrideInfo ? req.target : null) .setSourceGeneration(req.source.getGeneration()) .setRewriteToken(token) .setMaxBytesRewrittenPerCall(maxBytesRewrittenPerCall) diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/spi/StorageRpc.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/StorageRpc.java similarity index 72% rename from gcloud-java-storage/src/main/java/com/google/gcloud/spi/StorageRpc.java rename to gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/StorageRpc.java index e15a27114810..74f8171de87f 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/spi/StorageRpc.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/StorageRpc.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.storage.spi; import static com.google.common.base.MoreObjects.firstNonNull; @@ -138,15 +138,17 @@ class RewriteRequest { public final StorageObject source; public final Map sourceOptions; + public final boolean overrideInfo; public final StorageObject target; public final Map targetOptions; public final Long megabytesRewrittenPerCall; public RewriteRequest(StorageObject source, Map sourceOptions, - StorageObject target, Map targetOptions, + boolean overrideInfo, StorageObject target, Map targetOptions, Long megabytesRewrittenPerCall) { this.source = source; this.sourceOptions = sourceOptions; + this.overrideInfo = overrideInfo; this.target = target; this.targetOptions = targetOptions; this.megabytesRewrittenPerCall = megabytesRewrittenPerCall; @@ -163,6 +165,7 @@ public boolean equals(Object obj) { final RewriteRequest other = (RewriteRequest) obj; return Objects.equals(this.source, other.source) && Objects.equals(this.sourceOptions, other.sourceOptions) + && Objects.equals(this.overrideInfo, other.overrideInfo) && Objects.equals(this.target, other.target) && Objects.equals(this.targetOptions, other.targetOptions) && Objects.equals(this.megabytesRewrittenPerCall, other.megabytesRewrittenPerCall); @@ -170,7 +173,8 @@ public boolean equals(Object obj) { @Override public int hashCode() { - return Objects.hash(source, sourceOptions, target, targetOptions, megabytesRewrittenPerCall); + return Objects.hash(source, sourceOptions, overrideInfo, target, targetOptions, + megabytesRewrittenPerCall); } } @@ -217,57 +221,134 @@ public int hashCode() { } } - Bucket create(Bucket bucket, Map options) throws StorageException; + /** + * Creates a new bucket. + * + * @throws StorageException upon failure + */ + Bucket create(Bucket bucket, Map options); - StorageObject create(StorageObject object, InputStream content, Map options) - throws StorageException; + /** + * Creates a new storage object. + * + * @throws StorageException upon failure + */ + StorageObject create(StorageObject object, InputStream content, Map options); - Tuple> list(Map options) throws StorageException; + /** + * Lists the project's buckets. + * + * @throws StorageException upon failure + */ + Tuple> list(Map options); - Tuple> list(String bucket, Map options) - throws StorageException; + /** + * Lists the bucket's blobs. + * + * @throws StorageException upon failure + */ + Tuple> list(String bucket, Map options); /** * Returns the requested bucket or {@code null} if not found. * * @throws StorageException upon failure */ - Bucket get(Bucket bucket, Map options) throws StorageException; + Bucket get(Bucket bucket, Map options); /** * Returns the requested storage object or {@code null} if not found. * * @throws StorageException upon failure */ - StorageObject get(StorageObject object, Map options) - throws StorageException; + StorageObject get(StorageObject object, Map options); - Bucket patch(Bucket bucket, Map options) throws StorageException; + /** + * Updates bucket information. + * + * @throws StorageException upon failure + */ + Bucket patch(Bucket bucket, Map options); - StorageObject patch(StorageObject storageObject, Map options) - throws StorageException; + /** + * Updates the storage object's information. Original metadata are merged with metadata in the + * provided {@code storageObject}. + * + * @throws StorageException upon failure + */ + StorageObject patch(StorageObject storageObject, Map options); - boolean delete(Bucket bucket, Map options) throws StorageException; + /** + * Deletes the requested bucket. + * + * @return {@code true} if the bucket was deleted, {@code false} if it was not found + * @throws StorageException upon failure + */ + boolean delete(Bucket bucket, Map options); - boolean delete(StorageObject object, Map options) throws StorageException; + /** + * Deletes the requested storage object. + * + * @return {@code true} if the storage object was deleted, {@code false} if it was not found + * @throws StorageException upon failure + */ + boolean delete(StorageObject object, Map options); - BatchResponse batch(BatchRequest request) throws StorageException; + /** + * Sends a batch request. + * + * @throws StorageException upon failure + */ + BatchResponse batch(BatchRequest request); + /** + * Sends a compose request. + * + * @throws StorageException upon failure + */ StorageObject compose(Iterable sources, StorageObject target, - Map targetOptions) throws StorageException; + Map targetOptions); - byte[] load(StorageObject storageObject, Map options) - throws StorageException; + /** + * Reads all the bytes from a storage object. + * + * @throws StorageException upon failure + */ + byte[] load(StorageObject storageObject, Map options); - Tuple read(StorageObject from, Map options, long position, int bytes) - throws StorageException; + /** + * Reads the given amount of bytes from a storage object at the given position. + * + * @throws StorageException upon failure + */ + Tuple read(StorageObject from, Map options, long position, int bytes); - String open(StorageObject object, Map options) throws StorageException; + /** + * Opens a resumable upload channel for a given storage object. + * + * @throws StorageException upon failure + */ + String open(StorageObject object, Map options); + /** + * Writes the provided bytes to a storage object at the provided location. + * + * @throws StorageException upon failure + */ void write(String uploadId, byte[] toWrite, int toWriteOffset, long destOffset, int length, - boolean last) throws StorageException; + boolean last); - RewriteResponse openRewrite(RewriteRequest rewriteRequest) throws StorageException; + /** + * Sends a rewrite request to open a rewrite channel. + * + * @throws StorageException upon failure + */ + RewriteResponse openRewrite(RewriteRequest rewriteRequest); - RewriteResponse continueRewrite(RewriteResponse previousResponse) throws StorageException; + /** + * Continues rewriting on an already open rewrite channel. + * + * @throws StorageException upon failure + */ + RewriteResponse continueRewrite(RewriteResponse previousResponse); } diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/spi/StorageRpcFactory.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/StorageRpcFactory.java similarity index 91% rename from gcloud-java-storage/src/main/java/com/google/gcloud/spi/StorageRpcFactory.java rename to gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/StorageRpcFactory.java index f4959d617d17..19b98e6273db 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/spi/StorageRpcFactory.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/spi/StorageRpcFactory.java @@ -14,8 +14,9 @@ * limitations under the License. */ -package com.google.gcloud.spi; +package com.google.gcloud.storage.spi; +import com.google.gcloud.spi.ServiceRpcFactory; import com.google.gcloud.storage.StorageOptions; /** diff --git a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/testing/RemoteGcsHelper.java b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/testing/RemoteGcsHelper.java index 024aa04eba1b..1287ede746d5 100644 --- a/gcloud-java-storage/src/main/java/com/google/gcloud/storage/testing/RemoteGcsHelper.java +++ b/gcloud-java-storage/src/main/java/com/google/gcloud/storage/testing/RemoteGcsHelper.java @@ -20,6 +20,7 @@ import com.google.gcloud.RetryParams; import com.google.gcloud.storage.BlobInfo; import com.google.gcloud.storage.Storage; +import com.google.gcloud.storage.Storage.BlobListOption; import com.google.gcloud.storage.StorageException; import com.google.gcloud.storage.StorageOptions; @@ -173,8 +174,8 @@ public DeleteBucketTask(Storage storage, String bucket) { @Override public Boolean call() { while (true) { - for (BlobInfo info : storage.list(bucket).values()) { - storage.delete(bucket, info.name()); + for (BlobInfo info : storage.list(bucket, BlobListOption.versions(true)).values()) { + storage.delete(info.blobId()); } try { storage.delete(bucket); diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobInfoTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobInfoTest.java index a1cc01f4287c..029181c6c07b 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobInfoTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobInfoTest.java @@ -20,7 +20,11 @@ import static com.google.gcloud.storage.Acl.Role.READER; import static com.google.gcloud.storage.Acl.Role.WRITER; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import com.google.api.services.storage.model.StorageObject; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import com.google.gcloud.storage.Acl.Project; @@ -28,6 +32,7 @@ import org.junit.Test; +import java.math.BigInteger; import java.util.List; import java.util.Map; @@ -76,6 +81,10 @@ public class BlobInfoTest { .size(SIZE) .updateTime(UPDATE_TIME) .build(); + private static final BlobInfo DIRECTORY_INFO = BlobInfo.builder("b", "n/") + .size(0L) + .isDirectory(true) + .build(); @Test public void testToBuilder() { @@ -118,6 +127,30 @@ public void testBuilder() { assertEquals(SELF_LINK, BLOB_INFO.selfLink()); assertEquals(SIZE, BLOB_INFO.size()); assertEquals(UPDATE_TIME, BLOB_INFO.updateTime()); + assertFalse(BLOB_INFO.isDirectory()); + assertEquals("b", DIRECTORY_INFO.bucket()); + assertEquals("n/", DIRECTORY_INFO.name()); + assertNull(DIRECTORY_INFO.acl()); + assertNull(DIRECTORY_INFO.componentCount()); + assertNull(DIRECTORY_INFO.contentType()); + assertNull(DIRECTORY_INFO.cacheControl()); + assertNull(DIRECTORY_INFO.contentDisposition()); + assertNull(DIRECTORY_INFO.contentEncoding()); + assertNull(DIRECTORY_INFO.contentLanguage()); + assertNull(DIRECTORY_INFO.crc32c()); + assertNull(DIRECTORY_INFO.deleteTime()); + assertNull(DIRECTORY_INFO.etag()); + assertNull(DIRECTORY_INFO.generation()); + assertNull(DIRECTORY_INFO.id()); + assertNull(DIRECTORY_INFO.md5()); + assertNull(DIRECTORY_INFO.mediaLink()); + assertNull(DIRECTORY_INFO.metadata()); + assertNull(DIRECTORY_INFO.metageneration()); + assertNull(DIRECTORY_INFO.owner()); + assertNull(DIRECTORY_INFO.selfLink()); + assertEquals(0L, (long) DIRECTORY_INFO.size()); + assertNull(DIRECTORY_INFO.updateTime()); + assertTrue(DIRECTORY_INFO.isDirectory()); } private void compareBlobs(BlobInfo expected, BlobInfo value) { @@ -151,6 +184,35 @@ public void testToPbAndFromPb() { compareBlobs(BLOB_INFO, BlobInfo.fromPb(BLOB_INFO.toPb())); BlobInfo blobInfo = BlobInfo.builder(BlobId.of("b", "n")).build(); compareBlobs(blobInfo, BlobInfo.fromPb(blobInfo.toPb())); + StorageObject object = new StorageObject() + .setName("n/") + .setBucket("b") + .setSize(BigInteger.ZERO) + .set("isDirectory", true); + blobInfo = BlobInfo.fromPb(object); + assertEquals("b", blobInfo.bucket()); + assertEquals("n/", blobInfo.name()); + assertNull(blobInfo.acl()); + assertNull(blobInfo.componentCount()); + assertNull(blobInfo.contentType()); + assertNull(blobInfo.cacheControl()); + assertNull(blobInfo.contentDisposition()); + assertNull(blobInfo.contentEncoding()); + assertNull(blobInfo.contentLanguage()); + assertNull(blobInfo.crc32c()); + assertNull(blobInfo.deleteTime()); + assertNull(blobInfo.etag()); + assertNull(blobInfo.generation()); + assertNull(blobInfo.id()); + assertNull(blobInfo.md5()); + assertNull(blobInfo.mediaLink()); + assertNull(blobInfo.metadata()); + assertNull(blobInfo.metageneration()); + assertNull(blobInfo.owner()); + assertNull(blobInfo.selfLink()); + assertEquals(0L, (long) blobInfo.size()); + assertNull(blobInfo.updateTime()); + assertTrue(blobInfo.isDirectory()); } @Test diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobReadChannelTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobReadChannelTest.java index ffb37e8c5032..1b0f36a864a2 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobReadChannelTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobReadChannelTest.java @@ -30,8 +30,8 @@ import com.google.gcloud.ReadChannel; import com.google.gcloud.RestorableState; import com.google.gcloud.RetryParams; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpcFactory; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpcFactory; import org.junit.After; import org.junit.Before; @@ -39,6 +39,7 @@ import java.io.IOException; import java.nio.ByteBuffer; +import java.nio.channels.ClosedChannelException; import java.util.Arrays; import java.util.Map; import java.util.Random; @@ -156,15 +157,15 @@ public void testClose() { } @Test - public void testReadClosed() { + public void testReadClosed() throws IOException { replay(storageRpcMock); reader = new BlobReadChannel(options, BLOB_ID, EMPTY_RPC_OPTIONS); reader.close(); try { ByteBuffer readBuffer = ByteBuffer.allocate(DEFAULT_CHUNK_SIZE); reader.read(readBuffer); - fail("Expected BlobReadChannel read to throw IOException"); - } catch (IOException ex) { + fail("Expected BlobReadChannel read to throw ClosedChannelException"); + } catch (ClosedChannelException ex) { // expected } } diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobTest.java index c7508593f8c9..d6c97ca9ca03 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobTest.java @@ -95,6 +95,10 @@ public class BlobTest { .updateTime(UPDATE_TIME) .build(); private static final BlobInfo BLOB_INFO = BlobInfo.builder("b", "n").metageneration(42L).build(); + private static final BlobInfo DIRECTORY_INFO = BlobInfo.builder("b", "n/") + .size(0L) + .isDirectory(true) + .build(); private Storage storage; private Blob blob; @@ -229,6 +233,7 @@ public void testCopyToBucket() throws Exception { assertEquals(copyWriter, returnedCopyWriter); assertEquals(capturedCopyRequest.getValue().source(), blob.blobId()); assertEquals(capturedCopyRequest.getValue().target(), target); + assertFalse(capturedCopyRequest.getValue().overrideInfo()); assertTrue(capturedCopyRequest.getValue().sourceOptions().isEmpty()); assertTrue(capturedCopyRequest.getValue().targetOptions().isEmpty()); } @@ -247,6 +252,7 @@ public void testCopyTo() throws Exception { assertEquals(copyWriter, returnedCopyWriter); assertEquals(capturedCopyRequest.getValue().source(), blob.blobId()); assertEquals(capturedCopyRequest.getValue().target(), target); + assertFalse(capturedCopyRequest.getValue().overrideInfo()); assertTrue(capturedCopyRequest.getValue().sourceOptions().isEmpty()); assertTrue(capturedCopyRequest.getValue().targetOptions().isEmpty()); } @@ -254,9 +260,9 @@ public void testCopyTo() throws Exception { @Test public void testCopyToBlobId() throws Exception { initializeExpectedBlob(2); + BlobInfo target = BlobInfo.builder(BlobId.of("bt", "nt")).build(); BlobId targetId = BlobId.of("bt", "nt"); CopyWriter copyWriter = createMock(CopyWriter.class); - BlobInfo target = BlobInfo.builder(targetId).build(); Capture capturedCopyRequest = Capture.newInstance(); expect(storage.options()).andReturn(mockOptions); expect(storage.copy(capture(capturedCopyRequest))).andReturn(copyWriter); @@ -266,6 +272,7 @@ public void testCopyToBlobId() throws Exception { assertEquals(copyWriter, returnedCopyWriter); assertEquals(capturedCopyRequest.getValue().source(), blob.blobId()); assertEquals(capturedCopyRequest.getValue().target(), target); + assertFalse(capturedCopyRequest.getValue().overrideInfo()); assertTrue(capturedCopyRequest.getValue().sourceOptions().isEmpty()); assertTrue(capturedCopyRequest.getValue().targetOptions().isEmpty()); } @@ -305,18 +312,20 @@ public void testSignUrl() throws Exception { @Test public void testToBuilder() { - expect(storage.options()).andReturn(mockOptions).times(4); + expect(storage.options()).andReturn(mockOptions).times(6); replay(storage); Blob fullBlob = new Blob(storage, new BlobInfo.BuilderImpl(FULL_BLOB_INFO)); assertEquals(fullBlob, fullBlob.toBuilder().build()); Blob simpleBlob = new Blob(storage, new BlobInfo.BuilderImpl(BLOB_INFO)); assertEquals(simpleBlob, simpleBlob.toBuilder().build()); + Blob directory = new Blob(storage, new BlobInfo.BuilderImpl(DIRECTORY_INFO)); + assertEquals(directory, directory.toBuilder().build()); } @Test public void testBuilder() { initializeExpectedBlob(4); - expect(storage.options()).andReturn(mockOptions).times(2); + expect(storage.options()).andReturn(mockOptions).times(4); replay(storage); Blob.Builder builder = new Blob.Builder(new Blob(storage, new BlobInfo.BuilderImpl(BLOB_INFO))); Blob blob = builder.acl(ACL) @@ -360,5 +369,33 @@ public void testBuilder() { assertEquals(SELF_LINK, blob.selfLink()); assertEquals(SIZE, blob.size()); assertEquals(UPDATE_TIME, blob.updateTime()); + assertFalse(blob.isDirectory()); + builder = new Blob.Builder(new Blob(storage, new BlobInfo.BuilderImpl(DIRECTORY_INFO))); + blob = builder.blobId(BlobId.of("b", "n/")) + .isDirectory(true) + .size(0L) + .build(); + assertEquals("b", blob.bucket()); + assertEquals("n/", blob.name()); + assertNull(blob.acl()); + assertNull(blob.componentCount()); + assertNull(blob.contentType()); + assertNull(blob.cacheControl()); + assertNull(blob.contentDisposition()); + assertNull(blob.contentEncoding()); + assertNull(blob.contentLanguage()); + assertNull(blob.crc32c()); + assertNull(blob.deleteTime()); + assertNull(blob.etag()); + assertNull(blob.id()); + assertNull(blob.md5()); + assertNull(blob.mediaLink()); + assertNull(blob.metadata()); + assertNull(blob.metageneration()); + assertNull(blob.owner()); + assertNull(blob.selfLink()); + assertEquals(0L, (long) blob.size()); + assertNull(blob.updateTime()); + assertTrue(blob.isDirectory()); } } diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobWriteChannelTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobWriteChannelTest.java index e499f6b9de52..18ec64a9575f 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobWriteChannelTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BlobWriteChannelTest.java @@ -34,8 +34,8 @@ import com.google.gcloud.RestorableState; import com.google.gcloud.RetryParams; import com.google.gcloud.WriteChannel; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpcFactory; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpcFactory; import org.easymock.Capture; import org.easymock.CaptureType; diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BucketTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BucketTest.java index aac8a79d3a47..53056c39c0dc 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BucketTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/BucketTest.java @@ -42,7 +42,9 @@ import org.easymock.Capture; import org.junit.After; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; +import org.junit.rules.ExpectedException; import java.io.ByteArrayInputStream; import java.io.InputStream; @@ -99,6 +101,9 @@ public class BucketTest { private Bucket expectedBucket; private Iterable blobResults; + @Rule + public ExpectedException thrown = ExpectedException.none(); + @Before public void setUp() { storage = createStrictMock(Storage.class); @@ -288,19 +293,85 @@ public void testCreate() throws Exception { } @Test - public void testCreateNullContentType() throws Exception { + public void testCreateNoContentType() throws Exception { initializeExpectedBucket(5); - BlobInfo info = BlobInfo.builder("b", "n").contentType(Storage.DEFAULT_CONTENT_TYPE).build(); + BlobInfo info = BlobInfo.builder("b", "n").build(); Blob expectedBlob = new Blob(serviceMockReturnsOptions, new BlobInfo.BuilderImpl(info)); byte[] content = {0xD, 0xE, 0xA, 0xD}; expect(storage.options()).andReturn(mockOptions); expect(storage.create(info, content)).andReturn(expectedBlob); replay(storage); initializeBucket(); - Blob blob = bucket.create("n", content, null); + Blob blob = bucket.create("n", content); assertEquals(expectedBlob, blob); } + @Test + public void testCreateWithOptions() throws Exception { + initializeExpectedBucket(5); + BlobInfo info = BlobInfo.builder(BlobId.of("b", "n", 42L)) + .contentType(CONTENT_TYPE) + .metageneration(24L) + .build(); + Blob expectedBlob = new Blob(serviceMockReturnsOptions, new BlobInfo.BuilderImpl(info)); + byte[] content = {0xD, 0xE, 0xA, 0xD}; + Storage.PredefinedAcl acl = Storage.PredefinedAcl.ALL_AUTHENTICATED_USERS; + expect(storage.options()).andReturn(mockOptions); + expect(storage.create(info, content, Storage.BlobTargetOption.generationMatch(), + Storage.BlobTargetOption.metagenerationMatch(), + Storage.BlobTargetOption.predefinedAcl(acl))).andReturn(expectedBlob); + replay(storage); + initializeBucket(); + Blob blob = bucket.create("n", content, CONTENT_TYPE, + Bucket.BlobTargetOption.generationMatch(42L), + Bucket.BlobTargetOption.metagenerationMatch(24L), + Bucket.BlobTargetOption.predefinedAcl(acl)); + assertEquals(expectedBlob, blob); + } + + @Test + public void testCreateNotExists() throws Exception { + initializeExpectedBucket(5); + BlobInfo info = BlobInfo.builder(BlobId.of("b", "n", 0L)).contentType(CONTENT_TYPE).build(); + Blob expectedBlob = new Blob(serviceMockReturnsOptions, new BlobInfo.BuilderImpl(info)); + byte[] content = {0xD, 0xE, 0xA, 0xD}; + expect(storage.options()).andReturn(mockOptions); + expect(storage.create(info, content, Storage.BlobTargetOption.generationMatch())) + .andReturn(expectedBlob); + replay(storage); + initializeBucket(); + Blob blob = bucket.create("n", content, CONTENT_TYPE, Bucket.BlobTargetOption.doesNotExist()); + assertEquals(expectedBlob, blob); + } + + @Test + public void testCreateWithWrongGenerationOptions() throws Exception { + initializeExpectedBucket(4); + expect(storage.options()).andReturn(mockOptions); + replay(storage); + initializeBucket(); + byte[] content = {0xD, 0xE, 0xA, 0xD}; + thrown.expect(IllegalArgumentException.class); + thrown.expectMessage( + "Only one option of generationMatch, doesNotExist or generationNotMatch can be provided"); + bucket.create("n", content, CONTENT_TYPE, Bucket.BlobTargetOption.generationMatch(42L), + Bucket.BlobTargetOption.generationNotMatch(24L)); + } + + @Test + public void testCreateWithWrongMetagenerationOptions() throws Exception { + initializeExpectedBucket(4); + expect(storage.options()).andReturn(mockOptions); + replay(storage); + initializeBucket(); + byte[] content = {0xD, 0xE, 0xA, 0xD}; + thrown.expect(IllegalArgumentException.class); + thrown.expectMessage( + "metagenerationMatch and metagenerationNotMatch options can not be both provided"); + bucket.create("n", content, CONTENT_TYPE, Bucket.BlobTargetOption.metagenerationMatch(42L), + Bucket.BlobTargetOption.metagenerationNotMatch(24L)); + } + @Test public void testCreateFromStream() throws Exception { initializeExpectedBucket(5); @@ -317,9 +388,9 @@ public void testCreateFromStream() throws Exception { } @Test - public void testCreateFromStreamNullContentType() throws Exception { + public void testCreateFromStreamNoContentType() throws Exception { initializeExpectedBucket(5); - BlobInfo info = BlobInfo.builder("b", "n").contentType(Storage.DEFAULT_CONTENT_TYPE).build(); + BlobInfo info = BlobInfo.builder("b", "n").build(); Blob expectedBlob = new Blob(serviceMockReturnsOptions, new BlobInfo.BuilderImpl(info)); byte[] content = {0xD, 0xE, 0xA, 0xD}; InputStream streamContent = new ByteArrayInputStream(content); @@ -327,10 +398,84 @@ public void testCreateFromStreamNullContentType() throws Exception { expect(storage.create(info, streamContent)).andReturn(expectedBlob); replay(storage); initializeBucket(); - Blob blob = bucket.create("n", streamContent, null); + Blob blob = bucket.create("n", streamContent); + assertEquals(expectedBlob, blob); + } + + @Test + public void testCreateFromStreamWithOptions() throws Exception { + initializeExpectedBucket(5); + BlobInfo info = BlobInfo.builder(BlobId.of("b", "n", 42L)) + .contentType(CONTENT_TYPE) + .metageneration(24L) + .crc32c("crc") + .md5("md5") + .build(); + Blob expectedBlob = new Blob(serviceMockReturnsOptions, new BlobInfo.BuilderImpl(info)); + byte[] content = {0xD, 0xE, 0xA, 0xD}; + Storage.PredefinedAcl acl = Storage.PredefinedAcl.ALL_AUTHENTICATED_USERS; + InputStream streamContent = new ByteArrayInputStream(content); + expect(storage.options()).andReturn(mockOptions); + expect(storage.create(info, streamContent, Storage.BlobWriteOption.generationMatch(), + Storage.BlobWriteOption.metagenerationMatch(), Storage.BlobWriteOption.predefinedAcl(acl), + Storage.BlobWriteOption.crc32cMatch(), Storage.BlobWriteOption.md5Match())) + .andReturn(expectedBlob); + replay(storage); + initializeBucket(); + Blob blob = bucket.create("n", streamContent, CONTENT_TYPE, + Bucket.BlobWriteOption.generationMatch(42L), + Bucket.BlobWriteOption.metagenerationMatch(24L), Bucket.BlobWriteOption.predefinedAcl(acl), + Bucket.BlobWriteOption.crc32cMatch("crc"), Bucket.BlobWriteOption.md5Match("md5")); assertEquals(expectedBlob, blob); } + @Test + public void testCreateFromStreamNotExists() throws Exception { + initializeExpectedBucket(5); + BlobInfo info = BlobInfo.builder(BlobId.of("b", "n", 0L)).contentType(CONTENT_TYPE).build(); + Blob expectedBlob = new Blob(serviceMockReturnsOptions, new BlobInfo.BuilderImpl(info)); + byte[] content = {0xD, 0xE, 0xA, 0xD}; + InputStream streamContent = new ByteArrayInputStream(content); + expect(storage.options()).andReturn(mockOptions); + expect(storage.create(info, streamContent, Storage.BlobWriteOption.generationMatch())) + .andReturn(expectedBlob); + replay(storage); + initializeBucket(); + Blob blob = + bucket.create("n", streamContent, CONTENT_TYPE, Bucket.BlobWriteOption.doesNotExist()); + assertEquals(expectedBlob, blob); + } + + @Test + public void testCreateFromStreamWithWrongGenerationOptions() throws Exception { + initializeExpectedBucket(4); + expect(storage.options()).andReturn(mockOptions); + replay(storage); + initializeBucket(); + byte[] content = {0xD, 0xE, 0xA, 0xD}; + InputStream streamContent = new ByteArrayInputStream(content); + thrown.expect(IllegalArgumentException.class); + thrown.expectMessage( + "Only one option of generationMatch, doesNotExist or generationNotMatch can be provided"); + bucket.create("n", streamContent, CONTENT_TYPE, Bucket.BlobWriteOption.generationMatch(42L), + Bucket.BlobWriteOption.generationNotMatch(24L)); + } + + @Test + public void testCreateFromStreamWithWrongMetagenerationOptions() throws Exception { + initializeExpectedBucket(4); + expect(storage.options()).andReturn(mockOptions); + replay(storage); + initializeBucket(); + byte[] content = {0xD, 0xE, 0xA, 0xD}; + InputStream streamContent = new ByteArrayInputStream(content); + thrown.expect(IllegalArgumentException.class); + thrown.expectMessage( + "metagenerationMatch and metagenerationNotMatch options can not be both provided"); + bucket.create("n", streamContent, CONTENT_TYPE, Bucket.BlobWriteOption.metagenerationMatch(42L), + Bucket.BlobWriteOption.metagenerationNotMatch(24L)); + } + @Test public void testToBuilder() { expect(storage.options()).andReturn(mockOptions).times(4); diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/CopyRequestTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/CopyRequestTest.java index b7e8d14e53a1..9f8edfb84162 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/CopyRequestTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/CopyRequestTest.java @@ -18,6 +18,8 @@ import static com.google.gcloud.storage.Storage.PredefinedAcl.PUBLIC_READ; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; import com.google.common.collect.ImmutableList; import com.google.gcloud.storage.Storage.BlobSourceOption; @@ -53,6 +55,7 @@ public void testCopyRequest() { assertEquals(1, copyRequest1.sourceOptions().size()); assertEquals(BlobSourceOption.generationMatch(1), copyRequest1.sourceOptions().get(0)); assertEquals(TARGET_BLOB_INFO, copyRequest1.target()); + assertTrue(copyRequest1.overrideInfo()); assertEquals(1, copyRequest1.targetOptions().size()); assertEquals(BlobTargetOption.predefinedAcl(PUBLIC_READ), copyRequest1.targetOptions().get(0)); @@ -62,6 +65,7 @@ public void testCopyRequest() { .build(); assertEquals(SOURCE_BLOB_ID, copyRequest2.source()); assertEquals(BlobInfo.builder(TARGET_BLOB_ID).build(), copyRequest2.target()); + assertFalse(copyRequest2.overrideInfo()); Storage.CopyRequest copyRequest3 = Storage.CopyRequest.builder() .source(SOURCE_BLOB_ID) @@ -69,6 +73,7 @@ public void testCopyRequest() { .build(); assertEquals(SOURCE_BLOB_ID, copyRequest3.source()); assertEquals(TARGET_BLOB_INFO, copyRequest3.target()); + assertTrue(copyRequest3.overrideInfo()); assertEquals(ImmutableList.of(BlobTargetOption.predefinedAcl(PUBLIC_READ)), copyRequest3.targetOptions()); } @@ -78,52 +83,36 @@ public void testCopyRequestOf() { Storage.CopyRequest copyRequest1 = Storage.CopyRequest.of(SOURCE_BLOB_ID, TARGET_BLOB_INFO); assertEquals(SOURCE_BLOB_ID, copyRequest1.source()); assertEquals(TARGET_BLOB_INFO, copyRequest1.target()); + assertTrue(copyRequest1.overrideInfo()); Storage.CopyRequest copyRequest2 = Storage.CopyRequest.of(SOURCE_BLOB_ID, TARGET_BLOB_NAME); assertEquals(SOURCE_BLOB_ID, copyRequest2.source()); - assertEquals(BlobInfo.builder(SOURCE_BUCKET_NAME, TARGET_BLOB_NAME).build(), + assertEquals(BlobInfo.builder(BlobId.of(SOURCE_BUCKET_NAME, TARGET_BLOB_NAME)).build(), copyRequest2.target()); + assertFalse(copyRequest2.overrideInfo()); Storage.CopyRequest copyRequest3 = Storage.CopyRequest.of(SOURCE_BUCKET_NAME, SOURCE_BLOB_NAME, TARGET_BLOB_INFO); assertEquals(SOURCE_BLOB_ID, copyRequest3.source()); assertEquals(TARGET_BLOB_INFO, copyRequest3.target()); + assertTrue(copyRequest3.overrideInfo()); Storage.CopyRequest copyRequest4 = Storage.CopyRequest.of(SOURCE_BUCKET_NAME, SOURCE_BLOB_NAME, TARGET_BLOB_NAME); assertEquals(SOURCE_BLOB_ID, copyRequest4.source()); - assertEquals(BlobInfo.builder(SOURCE_BUCKET_NAME, TARGET_BLOB_NAME).build(), + assertEquals(BlobInfo.builder(BlobId.of(SOURCE_BUCKET_NAME, TARGET_BLOB_NAME)).build(), copyRequest4.target()); + assertFalse(copyRequest4.overrideInfo()); Storage.CopyRequest copyRequest5 = Storage.CopyRequest.of(SOURCE_BLOB_ID, TARGET_BLOB_ID); assertEquals(SOURCE_BLOB_ID, copyRequest5.source()); assertEquals(BlobInfo.builder(TARGET_BLOB_ID).build(), copyRequest5.target()); + assertFalse(copyRequest5.overrideInfo()); Storage.CopyRequest copyRequest6 = Storage.CopyRequest.of(SOURCE_BUCKET_NAME, SOURCE_BLOB_NAME, TARGET_BLOB_ID); assertEquals(SOURCE_BLOB_ID, copyRequest6.source()); assertEquals(BlobInfo.builder(TARGET_BLOB_ID).build(), copyRequest6.target()); - } - - @Test - public void testCopyRequestFail() { - thrown.expect(IllegalArgumentException.class); - Storage.CopyRequest.builder() - .source(SOURCE_BLOB_ID) - .target(BlobInfo.builder(TARGET_BLOB_ID).build()) - .build(); - } - - @Test - public void testCopyRequestOfBlobInfoFail() { - thrown.expect(IllegalArgumentException.class); - Storage.CopyRequest.of(SOURCE_BLOB_ID, BlobInfo.builder(TARGET_BLOB_ID).build()); - } - - @Test - public void testCopyRequestOfStringFail() { - thrown.expect(IllegalArgumentException.class); - Storage.CopyRequest.of( - SOURCE_BUCKET_NAME, SOURCE_BLOB_NAME, BlobInfo.builder(TARGET_BLOB_ID).build()); + assertFalse(copyRequest6.overrideInfo()); } } diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/CopyWriterTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/CopyWriterTest.java index 1b1ffd987de6..8ccb81688b65 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/CopyWriterTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/CopyWriterTest.java @@ -27,10 +27,10 @@ import com.google.common.collect.ImmutableMap; import com.google.gcloud.RestorableState; import com.google.gcloud.RetryParams; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpc.RewriteRequest; -import com.google.gcloud.spi.StorageRpc.RewriteResponse; -import com.google.gcloud.spi.StorageRpcFactory; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc.RewriteRequest; +import com.google.gcloud.storage.spi.StorageRpc.RewriteResponse; +import com.google.gcloud.storage.spi.StorageRpcFactory; import org.easymock.EasyMock; import org.junit.After; @@ -48,20 +48,29 @@ public class CopyWriterTest { private static final BlobId BLOB_ID = BlobId.of(SOURCE_BUCKET_NAME, SOURCE_BLOB_NAME); private static final BlobInfo BLOB_INFO = BlobInfo.builder(DESTINATION_BUCKET_NAME, DESTINATION_BLOB_NAME).build(); - private static final BlobInfo RESULT = + private static final BlobInfo RESULT_INFO = BlobInfo.builder(DESTINATION_BUCKET_NAME, DESTINATION_BLOB_NAME).contentType("type").build(); private static final Map EMPTY_OPTIONS = ImmutableMap.of(); - private static final RewriteRequest REQUEST = new StorageRpc.RewriteRequest(BLOB_ID.toPb(), - EMPTY_OPTIONS, BLOB_INFO.toPb(), EMPTY_OPTIONS, null); - private static final RewriteResponse RESPONSE = new StorageRpc.RewriteResponse(REQUEST, - null, 42L, false, "token", 21L); - private static final RewriteResponse RESPONSE_DONE = new StorageRpc.RewriteResponse(REQUEST, - RESULT.toPb(), 42L, true, "token", 42L); + private static final RewriteRequest REQUEST_WITH_OBJECT = + new StorageRpc.RewriteRequest(BLOB_ID.toPb(), EMPTY_OPTIONS, true, BLOB_INFO.toPb(), + EMPTY_OPTIONS, null); + private static final RewriteRequest REQUEST_WITHOUT_OBJECT = + new StorageRpc.RewriteRequest(BLOB_ID.toPb(), EMPTY_OPTIONS, false, BLOB_INFO.toPb(), + EMPTY_OPTIONS, null); + private static final RewriteResponse RESPONSE_WITH_OBJECT = new RewriteResponse( + REQUEST_WITH_OBJECT, null, 42L, false, "token", 21L); + private static final RewriteResponse RESPONSE_WITHOUT_OBJECT = new RewriteResponse( + REQUEST_WITHOUT_OBJECT, null, 42L, false, "token", 21L); + private static final RewriteResponse RESPONSE_WITH_OBJECT_DONE = + new RewriteResponse(REQUEST_WITH_OBJECT, RESULT_INFO.toPb(), 42L, true, "token", 42L); + private static final RewriteResponse RESPONSE_WITHOUT_OBJECT_DONE = + new RewriteResponse(REQUEST_WITHOUT_OBJECT, RESULT_INFO.toPb(), 42L, true, "token", 42L); private StorageOptions options; private StorageRpcFactory rpcFactoryMock; private StorageRpc storageRpcMock; private CopyWriter copyWriter; + private Blob result; @Before public void setUp() { @@ -75,6 +84,7 @@ public void setUp() { .serviceRpcFactory(rpcFactoryMock) .retryParams(RetryParams.noRetries()) .build(); + result = new Blob(options.service(), new BlobInfo.BuilderImpl(RESULT_INFO)); } @After @@ -83,41 +93,111 @@ public void tearDown() throws Exception { } @Test - public void testRewrite() { - EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE)).andReturn(RESPONSE_DONE); + public void testRewriteWithObject() { + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITH_OBJECT)) + .andReturn(RESPONSE_WITH_OBJECT_DONE); EasyMock.replay(storageRpcMock); - copyWriter = new CopyWriter(options, RESPONSE); - assertEquals(RESULT, copyWriter.result()); + copyWriter = new CopyWriter(options, RESPONSE_WITH_OBJECT); + assertEquals(result, copyWriter.result()); assertTrue(copyWriter.isDone()); assertEquals(42L, copyWriter.totalBytesCopied()); assertEquals(42L, copyWriter.blobSize()); } @Test - public void testRewriteMultipleRequests() { - EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE)).andReturn(RESPONSE); - EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE)).andReturn(RESPONSE_DONE); + public void testRewriteWithoutObject() { + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITHOUT_OBJECT)) + .andReturn(RESPONSE_WITHOUT_OBJECT_DONE); EasyMock.replay(storageRpcMock); - copyWriter = new CopyWriter(options, RESPONSE); - assertEquals(RESULT, copyWriter.result()); + copyWriter = new CopyWriter(options, RESPONSE_WITHOUT_OBJECT); + assertEquals(result, copyWriter.result()); assertTrue(copyWriter.isDone()); assertEquals(42L, copyWriter.totalBytesCopied()); assertEquals(42L, copyWriter.blobSize()); } @Test - public void testSaveAndRestore() { - EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE)).andReturn(RESPONSE); - EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE)).andReturn(RESPONSE_DONE); + public void testRewriteWithObjectMultipleRequests() { + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITH_OBJECT)) + .andReturn(RESPONSE_WITH_OBJECT); + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITH_OBJECT)) + .andReturn(RESPONSE_WITH_OBJECT_DONE); EasyMock.replay(storageRpcMock); - copyWriter = new CopyWriter(options, RESPONSE); + copyWriter = new CopyWriter(options, RESPONSE_WITH_OBJECT); + assertEquals(result, copyWriter.result()); + assertTrue(copyWriter.isDone()); + assertEquals(42L, copyWriter.totalBytesCopied()); + assertEquals(42L, copyWriter.blobSize()); + } + + @Test + public void testRewriteWithoutObjectMultipleRequests() { + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITHOUT_OBJECT)) + .andReturn(RESPONSE_WITHOUT_OBJECT); + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITHOUT_OBJECT)) + .andReturn(RESPONSE_WITHOUT_OBJECT_DONE); + EasyMock.replay(storageRpcMock); + copyWriter = new CopyWriter(options, RESPONSE_WITHOUT_OBJECT); + assertEquals(result, copyWriter.result()); + assertTrue(copyWriter.isDone()); + assertEquals(42L, copyWriter.totalBytesCopied()); + assertEquals(42L, copyWriter.blobSize()); + } + + @Test + public void testSaveAndRestoreWithObject() { + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITH_OBJECT)) + .andReturn(RESPONSE_WITH_OBJECT); + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITH_OBJECT)) + .andReturn(RESPONSE_WITH_OBJECT_DONE); + EasyMock.replay(storageRpcMock); + copyWriter = new CopyWriter(options, RESPONSE_WITH_OBJECT); + copyWriter.copyChunk(); + assertTrue(!copyWriter.isDone()); + assertEquals(21L, copyWriter.totalBytesCopied()); + assertEquals(42L, copyWriter.blobSize()); + RestorableState rewriterState = copyWriter.capture(); + CopyWriter restoredRewriter = rewriterState.restore(); + assertEquals(result, restoredRewriter.result()); + assertTrue(restoredRewriter.isDone()); + assertEquals(42L, restoredRewriter.totalBytesCopied()); + assertEquals(42L, restoredRewriter.blobSize()); + } + + @Test + public void testSaveAndRestoreWithoutObject() { + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITHOUT_OBJECT)) + .andReturn(RESPONSE_WITHOUT_OBJECT); + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITHOUT_OBJECT)) + .andReturn(RESPONSE_WITHOUT_OBJECT_DONE); + EasyMock.replay(storageRpcMock); + copyWriter = new CopyWriter(options, RESPONSE_WITHOUT_OBJECT); copyWriter.copyChunk(); assertTrue(!copyWriter.isDone()); assertEquals(21L, copyWriter.totalBytesCopied()); assertEquals(42L, copyWriter.blobSize()); RestorableState rewriterState = copyWriter.capture(); CopyWriter restoredRewriter = rewriterState.restore(); - assertEquals(RESULT, restoredRewriter.result()); + assertEquals(result, restoredRewriter.result()); + assertTrue(restoredRewriter.isDone()); + assertEquals(42L, restoredRewriter.totalBytesCopied()); + assertEquals(42L, restoredRewriter.blobSize()); + } + + @Test + public void testSaveAndRestoreWithResult() { + EasyMock.expect(storageRpcMock.continueRewrite(RESPONSE_WITH_OBJECT)) + .andReturn(RESPONSE_WITH_OBJECT_DONE); + EasyMock.replay(storageRpcMock); + copyWriter = new CopyWriter(options, RESPONSE_WITH_OBJECT); + copyWriter.copyChunk(); + assertEquals(result, copyWriter.result()); + assertTrue(copyWriter.isDone()); + assertEquals(42L, copyWriter.totalBytesCopied()); + assertEquals(42L, copyWriter.blobSize()); + RestorableState rewriterState = copyWriter.capture(); + CopyWriter restoredRewriter = rewriterState.restore(); + assertEquals(result, restoredRewriter.result()); assertTrue(restoredRewriter.isDone()); assertEquals(42L, restoredRewriter.totalBytesCopied()); assertEquals(42L, restoredRewriter.blobSize()); diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/OptionTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/OptionTest.java index 2703ddb401c5..5924174ab138 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/OptionTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/OptionTest.java @@ -18,7 +18,7 @@ import static org.junit.Assert.assertEquals; -import com.google.gcloud.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc; import org.junit.Test; diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/RemoteGcsHelperTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/RemoteGcsHelperTest.java index 2f56bbda7bd9..146922a9dae9 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/RemoteGcsHelperTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/RemoteGcsHelperTest.java @@ -21,6 +21,7 @@ import com.google.common.collect.ImmutableList; import com.google.gcloud.Page; +import com.google.gcloud.storage.Storage.BlobListOption; import com.google.gcloud.storage.testing.RemoteGcsHelper; import org.easymock.EasyMock; @@ -117,9 +118,10 @@ public Iterator iterateAll() { @Test public void testForceDelete() throws InterruptedException, ExecutionException { Storage storageMock = EasyMock.createMock(Storage.class); - EasyMock.expect(storageMock.list(BUCKET_NAME)).andReturn(blobPage); + EasyMock.expect(storageMock.list(BUCKET_NAME, BlobListOption.versions(true))) + .andReturn(blobPage); for (BlobInfo info : blobList) { - EasyMock.expect(storageMock.delete(BUCKET_NAME, info.name())).andReturn(true); + EasyMock.expect(storageMock.delete(info.blobId())).andReturn(true); } EasyMock.expect(storageMock.delete(BUCKET_NAME)).andReturn(true); EasyMock.replay(storageMock); @@ -130,9 +132,10 @@ public void testForceDelete() throws InterruptedException, ExecutionException { @Test public void testForceDeleteTimeout() throws InterruptedException, ExecutionException { Storage storageMock = EasyMock.createMock(Storage.class); - EasyMock.expect(storageMock.list(BUCKET_NAME)).andReturn(blobPage).anyTimes(); + EasyMock.expect(storageMock.list(BUCKET_NAME, BlobListOption.versions(true))) + .andReturn(blobPage).anyTimes(); for (BlobInfo info : blobList) { - EasyMock.expect(storageMock.delete(BUCKET_NAME, info.name())).andReturn(true).anyTimes(); + EasyMock.expect(storageMock.delete(info.blobId())).andReturn(true).anyTimes(); } EasyMock.expect(storageMock.delete(BUCKET_NAME)).andThrow(RETRYABLE_EXCEPTION).anyTimes(); EasyMock.replay(storageMock); @@ -143,9 +146,10 @@ public void testForceDeleteTimeout() throws InterruptedException, ExecutionExcep @Test public void testForceDeleteFail() throws InterruptedException, ExecutionException { Storage storageMock = EasyMock.createMock(Storage.class); - EasyMock.expect(storageMock.list(BUCKET_NAME)).andReturn(blobPage); + EasyMock.expect(storageMock.list(BUCKET_NAME, BlobListOption.versions(true))) + .andReturn(blobPage); for (BlobInfo info : blobList) { - EasyMock.expect(storageMock.delete(BUCKET_NAME, info.name())).andReturn(true); + EasyMock.expect(storageMock.delete(info.blobId())).andReturn(true); } EasyMock.expect(storageMock.delete(BUCKET_NAME)).andThrow(FATAL_EXCEPTION); EasyMock.replay(storageMock); @@ -160,9 +164,10 @@ public void testForceDeleteFail() throws InterruptedException, ExecutionExceptio @Test public void testForceDeleteNoTimeout() { Storage storageMock = EasyMock.createMock(Storage.class); - EasyMock.expect(storageMock.list(BUCKET_NAME)).andReturn(blobPage); + EasyMock.expect(storageMock.list(BUCKET_NAME, BlobListOption.versions(true))) + .andReturn(blobPage); for (BlobInfo info : blobList) { - EasyMock.expect(storageMock.delete(BUCKET_NAME, info.name())).andReturn(true); + EasyMock.expect(storageMock.delete(info.blobId())).andReturn(true); } EasyMock.expect(storageMock.delete(BUCKET_NAME)).andReturn(true); EasyMock.replay(storageMock); @@ -173,9 +178,10 @@ public void testForceDeleteNoTimeout() { @Test public void testForceDeleteNoTimeoutFail() { Storage storageMock = EasyMock.createMock(Storage.class); - EasyMock.expect(storageMock.list(BUCKET_NAME)).andReturn(blobPage); + EasyMock.expect(storageMock.list(BUCKET_NAME, BlobListOption.versions(true))) + .andReturn(blobPage); for (BlobInfo info : blobList) { - EasyMock.expect(storageMock.delete(BUCKET_NAME, info.name())).andReturn(true); + EasyMock.expect(storageMock.delete(info.blobId())).andReturn(true); } EasyMock.expect(storageMock.delete(BUCKET_NAME)).andThrow(FATAL_EXCEPTION); EasyMock.replay(storageMock); diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/SerializationTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/SerializationTest.java index c9b957bb936a..613cb81c3549 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/SerializationTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/SerializationTest.java @@ -16,31 +16,20 @@ package com.google.gcloud.storage; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotSame; - import com.google.common.collect.ImmutableMap; import com.google.gcloud.AuthCredentials; +import com.google.gcloud.BaseSerializationTest; import com.google.gcloud.PageImpl; import com.google.gcloud.ReadChannel; -import com.google.gcloud.RestorableState; -import com.google.gcloud.RetryParams; -import com.google.gcloud.WriteChannel; -import com.google.gcloud.spi.StorageRpc; +import com.google.gcloud.Restorable; import com.google.gcloud.storage.Acl.Project.ProjectRole; +import com.google.gcloud.storage.spi.StorageRpc; -import org.junit.Test; - -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.IOException; -import java.io.ObjectInputStream; -import java.io.ObjectOutputStream; import java.io.Serializable; import java.util.Collections; import java.util.Map; -public class SerializationTest { +public class SerializationTest extends BaseSerializationTest { private static final Storage STORAGE = StorageOptions.builder().projectId("p").build().service(); private static final Acl.Domain ACL_DOMAIN = new Acl.Domain("domain"); @@ -63,8 +52,9 @@ public class SerializationTest { Collections.>emptyList()); private static final PageImpl PAGE_RESULT = new PageImpl<>(null, "c", Collections.singletonList(BLOB)); + private static final StorageException STORAGE_EXCEPTION = new StorageException(42, "message"); private static final Storage.BlobListOption BLOB_LIST_OPTIONS = - Storage.BlobListOption.maxResults(100); + Storage.BlobListOption.pageSize(100); private static final Storage.BlobSourceOption BLOB_SOURCE_OPTIONS = Storage.BlobSourceOption.generationMatch(1); private static final Storage.BlobTargetOption BLOB_TARGET_OPTIONS = @@ -77,83 +67,32 @@ public class SerializationTest { Storage.BucketTargetOption.metagenerationNotMatch(); private static final Map EMPTY_RPC_OPTIONS = ImmutableMap.of(); - @Test - public void testServiceOptions() throws Exception { + @Override + protected Serializable[] serializableObjects() { StorageOptions options = StorageOptions.builder() .projectId("p1") .authCredentials(AuthCredentials.createForAppEngine()) .build(); - StorageOptions serializedCopy = serializeAndDeserialize(options); - assertEquals(options, serializedCopy); - - options = options.toBuilder() + StorageOptions otherOptions = options.toBuilder() .projectId("p2") - .retryParams(RetryParams.defaultInstance()) .authCredentials(null) - .pathDelimiter(":") .build(); - serializedCopy = serializeAndDeserialize(options); - assertEquals(options, serializedCopy); - } - - @Test - public void testModelAndRequests() throws Exception { - Serializable[] objects = {ACL_DOMAIN, ACL_GROUP, ACL_PROJECT_, ACL_USER, ACL_RAW, ACL, + return new Serializable[]{ACL_DOMAIN, ACL_GROUP, ACL_PROJECT_, ACL_USER, ACL_RAW, ACL, BLOB_INFO, BLOB, BUCKET_INFO, BUCKET, ORIGIN, CORS, BATCH_REQUEST, BATCH_RESPONSE, PAGE_RESULT, BLOB_LIST_OPTIONS, BLOB_SOURCE_OPTIONS, BLOB_TARGET_OPTIONS, - BUCKET_LIST_OPTIONS, BUCKET_SOURCE_OPTIONS, BUCKET_TARGET_OPTIONS}; - for (Serializable obj : objects) { - Object copy = serializeAndDeserialize(obj); - assertEquals(obj, obj); - assertEquals(obj, copy); - assertNotSame(obj, copy); - assertEquals(copy, copy); - } + BUCKET_LIST_OPTIONS, BUCKET_SOURCE_OPTIONS, BUCKET_TARGET_OPTIONS, STORAGE_EXCEPTION, + options, otherOptions}; } - @Test - public void testReadChannelState() throws IOException, ClassNotFoundException { - StorageOptions options = StorageOptions.builder() - .projectId("p2") - .retryParams(RetryParams.defaultInstance()) - .build(); + @Override + protected Restorable[] restorableObjects() { + StorageOptions options = StorageOptions.builder().projectId("p2").build(); ReadChannel reader = new BlobReadChannel(options, BlobId.of("b", "n"), EMPTY_RPC_OPTIONS); - RestorableState state = reader.capture(); - RestorableState deserializedState = serializeAndDeserialize(state); - assertEquals(state, deserializedState); - assertEquals(state.hashCode(), deserializedState.hashCode()); - assertEquals(state.toString(), deserializedState.toString()); - reader.close(); - } - - @Test - public void testWriteChannelState() throws IOException, ClassNotFoundException { - StorageOptions options = StorageOptions.builder() - .projectId("p2") - .retryParams(RetryParams.defaultInstance()) - .build(); // avoid closing when you don't want partial writes to GCS upon failure @SuppressWarnings("resource") BlobWriteChannel writer = new BlobWriteChannel(options, BlobInfo.builder(BlobId.of("b", "n")).build(), "upload-id"); - RestorableState state = writer.capture(); - RestorableState deserializedState = serializeAndDeserialize(state); - assertEquals(state, deserializedState); - assertEquals(state.hashCode(), deserializedState.hashCode()); - assertEquals(state.toString(), deserializedState.toString()); - } - - @SuppressWarnings("unchecked") - private T serializeAndDeserialize(T obj) - throws IOException, ClassNotFoundException { - ByteArrayOutputStream bytes = new ByteArrayOutputStream(); - try (ObjectOutputStream output = new ObjectOutputStream(bytes)) { - output.writeObject(obj); - } - try (ObjectInputStream input = - new ObjectInputStream(new ByteArrayInputStream(bytes.toByteArray()))) { - return (T) input.readObject(); - } + return new Restorable[]{reader, writer}; } } diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/StorageImplTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/StorageImplTest.java index f49938c5e9c1..3cc99e3bf884 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/StorageImplTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/StorageImplTest.java @@ -37,10 +37,10 @@ import com.google.gcloud.RetryParams; import com.google.gcloud.ServiceOptions; import com.google.gcloud.WriteChannel; -import com.google.gcloud.spi.StorageRpc; -import com.google.gcloud.spi.StorageRpc.Tuple; -import com.google.gcloud.spi.StorageRpcFactory; import com.google.gcloud.storage.Storage.CopyRequest; +import com.google.gcloud.storage.spi.StorageRpc; +import com.google.gcloud.storage.spi.StorageRpc.Tuple; +import com.google.gcloud.storage.spi.StorageRpcFactory; import org.easymock.Capture; import org.easymock.EasyMock; @@ -181,8 +181,8 @@ public class StorageImplTest { StorageRpc.Option.IF_SOURCE_GENERATION_MATCH, BLOB_SOURCE_GENERATION.value()); // Bucket list options - private static final Storage.BucketListOption BUCKET_LIST_MAX_RESULT = - Storage.BucketListOption.maxResults(42L); + private static final Storage.BucketListOption BUCKET_LIST_PAGE_SIZE = + Storage.BucketListOption.pageSize(42L); private static final Storage.BucketListOption BUCKET_LIST_PREFIX = Storage.BucketListOption.prefix("prefix"); private static final Storage.BucketListOption BUCKET_LIST_FIELDS = @@ -190,21 +190,24 @@ public class StorageImplTest { private static final Storage.BucketListOption BUCKET_LIST_EMPTY_FIELDS = Storage.BucketListOption.fields(); private static final Map BUCKET_LIST_OPTIONS = ImmutableMap.of( - StorageRpc.Option.MAX_RESULTS, BUCKET_LIST_MAX_RESULT.value(), + StorageRpc.Option.MAX_RESULTS, BUCKET_LIST_PAGE_SIZE.value(), StorageRpc.Option.PREFIX, BUCKET_LIST_PREFIX.value()); // Blob list options - private static final Storage.BlobListOption BLOB_LIST_MAX_RESULT = - Storage.BlobListOption.maxResults(42L); + private static final Storage.BlobListOption BLOB_LIST_PAGE_SIZE = + Storage.BlobListOption.pageSize(42L); private static final Storage.BlobListOption BLOB_LIST_PREFIX = Storage.BlobListOption.prefix("prefix"); private static final Storage.BlobListOption BLOB_LIST_FIELDS = Storage.BlobListOption.fields(Storage.BlobField.CONTENT_TYPE, Storage.BlobField.MD5HASH); + private static final Storage.BlobListOption BLOB_LIST_VERSIONS = + Storage.BlobListOption.versions(false); private static final Storage.BlobListOption BLOB_LIST_EMPTY_FIELDS = Storage.BlobListOption.fields(); private static final Map BLOB_LIST_OPTIONS = ImmutableMap.of( - StorageRpc.Option.MAX_RESULTS, BLOB_LIST_MAX_RESULT.value(), - StorageRpc.Option.PREFIX, BLOB_LIST_PREFIX.value()); + StorageRpc.Option.MAX_RESULTS, BLOB_LIST_PAGE_SIZE.value(), + StorageRpc.Option.PREFIX, BLOB_LIST_PREFIX.value(), + StorageRpc.Option.VERSIONS, BLOB_LIST_VERSIONS.value()); private static final String PRIVATE_KEY_STRING = "MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoG" + "BAL2xolH1zrISQ8+GzOV29BNjjzq4/HIP8Psd1+cZb81vDklSF+95wB250MSE0BDc81pvIMwj5OmIfLg1NY6uB" @@ -564,7 +567,7 @@ public void testListBucketsWithOptions() { EasyMock.replay(storageRpcMock); initializeService(); ImmutableList bucketList = ImmutableList.of(expectedBucket1, expectedBucket2); - Page page = storage.list(BUCKET_LIST_MAX_RESULT, BUCKET_LIST_PREFIX); + Page page = storage.list(BUCKET_LIST_PAGE_SIZE, BUCKET_LIST_PREFIX); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(bucketList.toArray(), Iterables.toArray(page.values(), Bucket.class)); } @@ -586,7 +589,8 @@ public void testListBucketsWithSelectedFields() { assertTrue(selector.contains("name")); assertTrue(selector.contains("acl")); assertTrue(selector.contains("location")); - assertEquals(24, selector.length()); + assertTrue(selector.contains("nextPageToken")); + assertEquals(38, selector.length()); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(bucketList.toArray(), Iterables.toArray(page.values(), Bucket.class)); } @@ -606,7 +610,8 @@ public void testListBucketsWithEmptyFields() { String selector = (String) capturedOptions.getValue().get(BLOB_LIST_FIELDS.rpcOption()); assertTrue(selector.contains("items")); assertTrue(selector.contains("name")); - assertEquals(11, selector.length()); + assertTrue(selector.contains("nextPageToken")); + assertEquals(25, selector.length()); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(bucketList.toArray(), Iterables.toArray(page.values(), Bucket.class)); } @@ -648,7 +653,8 @@ public void testListBlobsWithOptions() { EasyMock.replay(storageRpcMock); initializeService(); ImmutableList blobList = ImmutableList.of(expectedBlob1, expectedBlob2); - Page page = storage.list(BUCKET_NAME1, BLOB_LIST_MAX_RESULT, BLOB_LIST_PREFIX); + Page page = + storage.list(BUCKET_NAME1, BLOB_LIST_PAGE_SIZE, BLOB_LIST_PREFIX, BLOB_LIST_VERSIONS); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(blobList.toArray(), Iterables.toArray(page.values(), Blob.class)); } @@ -667,9 +673,9 @@ public void testListBlobsWithSelectedFields() { initializeService(); ImmutableList blobList = ImmutableList.of(expectedBlob1, expectedBlob2); Page page = - storage.list(BUCKET_NAME1, BLOB_LIST_MAX_RESULT, BLOB_LIST_PREFIX, BLOB_LIST_FIELDS); - assertEquals(BLOB_LIST_MAX_RESULT.value(), - capturedOptions.getValue().get(BLOB_LIST_MAX_RESULT.rpcOption())); + storage.list(BUCKET_NAME1, BLOB_LIST_PAGE_SIZE, BLOB_LIST_PREFIX, BLOB_LIST_FIELDS); + assertEquals(BLOB_LIST_PAGE_SIZE.value(), + capturedOptions.getValue().get(BLOB_LIST_PAGE_SIZE.rpcOption())); assertEquals(BLOB_LIST_PREFIX.value(), capturedOptions.getValue().get(BLOB_LIST_PREFIX.rpcOption())); String selector = (String) capturedOptions.getValue().get(BLOB_LIST_FIELDS.rpcOption()); @@ -678,7 +684,8 @@ public void testListBlobsWithSelectedFields() { assertTrue(selector.contains("name")); assertTrue(selector.contains("contentType")); assertTrue(selector.contains("md5Hash")); - assertEquals(38, selector.length()); + assertTrue(selector.contains("nextPageToken")); + assertEquals(52, selector.length()); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(blobList.toArray(), Iterables.toArray(page.values(), Blob.class)); } @@ -697,16 +704,33 @@ public void testListBlobsWithEmptyFields() { initializeService(); ImmutableList blobList = ImmutableList.of(expectedBlob1, expectedBlob2); Page page = - storage.list(BUCKET_NAME1, BLOB_LIST_MAX_RESULT, BLOB_LIST_PREFIX, BLOB_LIST_EMPTY_FIELDS); - assertEquals(BLOB_LIST_MAX_RESULT.value(), - capturedOptions.getValue().get(BLOB_LIST_MAX_RESULT.rpcOption())); + storage.list(BUCKET_NAME1, BLOB_LIST_PAGE_SIZE, BLOB_LIST_PREFIX, BLOB_LIST_EMPTY_FIELDS); + assertEquals(BLOB_LIST_PAGE_SIZE.value(), + capturedOptions.getValue().get(BLOB_LIST_PAGE_SIZE.rpcOption())); assertEquals(BLOB_LIST_PREFIX.value(), capturedOptions.getValue().get(BLOB_LIST_PREFIX.rpcOption())); String selector = (String) capturedOptions.getValue().get(BLOB_LIST_EMPTY_FIELDS.rpcOption()); assertTrue(selector.contains("items")); assertTrue(selector.contains("bucket")); assertTrue(selector.contains("name")); - assertEquals(18, selector.length()); + assertTrue(selector.contains("nextPageToken")); + assertEquals(32, selector.length()); + assertEquals(cursor, page.nextPageCursor()); + assertArrayEquals(blobList.toArray(), Iterables.toArray(page.values(), Blob.class)); + } + + @Test + public void testListBlobsCurrentDirectory() { + String cursor = "cursor"; + Map options = ImmutableMap.of(StorageRpc.Option.DELIMITER, "/"); + ImmutableList blobInfoList = ImmutableList.of(BLOB_INFO1, BLOB_INFO2); + Tuple> result = + Tuple.of(cursor, Iterables.transform(blobInfoList, BlobInfo.INFO_TO_PB_FUNCTION)); + EasyMock.expect(storageRpcMock.list(BUCKET_NAME1, options)).andReturn(result); + EasyMock.replay(storageRpcMock); + initializeService(); + ImmutableList blobList = ImmutableList.of(expectedBlob1, expectedBlob2); + Page page = storage.list(BUCKET_NAME1, Storage.BlobListOption.currentDirectory()); assertEquals(cursor, page.nextPageCursor()); assertArrayEquals(blobList.toArray(), Iterables.toArray(page.values(), Blob.class)); } @@ -842,7 +866,7 @@ public void testComposeWithOptions() { public void testCopy() { CopyRequest request = Storage.CopyRequest.of(BLOB_INFO1.blobId(), BLOB_INFO2.blobId()); StorageRpc.RewriteRequest rpcRequest = new StorageRpc.RewriteRequest(request.source().toPb(), - EMPTY_RPC_OPTIONS, request.target().toPb(), EMPTY_RPC_OPTIONS, null); + EMPTY_RPC_OPTIONS, false, BLOB_INFO2.toPb(), EMPTY_RPC_OPTIONS, null); StorageRpc.RewriteResponse rpcResponse = new StorageRpc.RewriteResponse(rpcRequest, null, 42L, false, "token", 21L); EasyMock.expect(storageRpcMock.openRewrite(rpcRequest)).andReturn(rpcResponse); @@ -862,7 +886,7 @@ public void testCopyWithOptions() { .target(BLOB_INFO1, BLOB_TARGET_GENERATION, BLOB_TARGET_METAGENERATION) .build(); StorageRpc.RewriteRequest rpcRequest = new StorageRpc.RewriteRequest(request.source().toPb(), - BLOB_SOURCE_OPTIONS_COPY, request.target().toPb(), BLOB_TARGET_OPTIONS_COMPOSE, null); + BLOB_SOURCE_OPTIONS_COPY, true, request.target().toPb(), BLOB_TARGET_OPTIONS_COMPOSE, null); StorageRpc.RewriteResponse rpcResponse = new StorageRpc.RewriteResponse(rpcRequest, null, 42L, false, "token", 21L); EasyMock.expect(storageRpcMock.openRewrite(rpcRequest)).andReturn(rpcResponse); @@ -882,7 +906,7 @@ public void testCopyWithOptionsFromBlobId() { .target(BLOB_INFO1, BLOB_TARGET_GENERATION, BLOB_TARGET_METAGENERATION) .build(); StorageRpc.RewriteRequest rpcRequest = new StorageRpc.RewriteRequest(request.source().toPb(), - BLOB_SOURCE_OPTIONS_COPY, request.target().toPb(), BLOB_TARGET_OPTIONS_COMPOSE, null); + BLOB_SOURCE_OPTIONS_COPY, true, request.target().toPb(), BLOB_TARGET_OPTIONS_COMPOSE, null); StorageRpc.RewriteResponse rpcResponse = new StorageRpc.RewriteResponse(rpcRequest, null, 42L, false, "token", 21L); EasyMock.expect(storageRpcMock.openRewrite(rpcRequest)).andReturn(rpcResponse); @@ -898,7 +922,7 @@ public void testCopyWithOptionsFromBlobId() { public void testCopyMultipleRequests() { CopyRequest request = Storage.CopyRequest.of(BLOB_INFO1.blobId(), BLOB_INFO2.blobId()); StorageRpc.RewriteRequest rpcRequest = new StorageRpc.RewriteRequest(request.source().toPb(), - EMPTY_RPC_OPTIONS, request.target().toPb(), EMPTY_RPC_OPTIONS, null); + EMPTY_RPC_OPTIONS, false, BLOB_INFO2.toPb(), EMPTY_RPC_OPTIONS, null); StorageRpc.RewriteResponse rpcResponse1 = new StorageRpc.RewriteResponse(rpcRequest, null, 42L, false, "token", 21L); StorageRpc.RewriteResponse rpcResponse2 = new StorageRpc.RewriteResponse(rpcRequest, @@ -911,7 +935,7 @@ public void testCopyMultipleRequests() { assertEquals(42L, writer.blobSize()); assertEquals(21L, writer.totalBytesCopied()); assertTrue(!writer.isDone()); - assertEquals(BLOB_INFO1, writer.result()); + assertEquals(expectedBlob1, writer.result()); assertTrue(writer.isDone()); assertEquals(42L, writer.totalBytesCopied()); assertEquals(42L, writer.blobSize()); diff --git a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/ITStorageTest.java b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/it/ITStorageTest.java similarity index 78% rename from gcloud-java-storage/src/test/java/com/google/gcloud/storage/ITStorageTest.java rename to gcloud-java-storage/src/test/java/com/google/gcloud/storage/it/ITStorageTest.java index dffcc366036a..13d768442c34 100644 --- a/gcloud-java-storage/src/test/java/com/google/gcloud/storage/ITStorageTest.java +++ b/gcloud-java-storage/src/test/java/com/google/gcloud/storage/it/ITStorageTest.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.google.gcloud.storage; +package com.google.gcloud.storage.it; import static java.nio.charset.StandardCharsets.UTF_8; import static org.junit.Assert.assertArrayEquals; @@ -25,15 +25,28 @@ import static org.junit.Assert.assertTrue; import static org.junit.Assert.fail; -import com.google.api.client.util.Lists; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; +import com.google.common.collect.ImmutableSet; +import com.google.common.collect.Iterators; +import com.google.common.collect.Lists; import com.google.gcloud.Page; import com.google.gcloud.ReadChannel; import com.google.gcloud.RestorableState; import com.google.gcloud.WriteChannel; +import com.google.gcloud.storage.BatchRequest; +import com.google.gcloud.storage.BatchResponse; +import com.google.gcloud.storage.Blob; +import com.google.gcloud.storage.BlobId; +import com.google.gcloud.storage.BlobInfo; +import com.google.gcloud.storage.Bucket; +import com.google.gcloud.storage.BucketInfo; +import com.google.gcloud.storage.CopyWriter; +import com.google.gcloud.storage.HttpMethod; +import com.google.gcloud.storage.Storage; import com.google.gcloud.storage.Storage.BlobField; import com.google.gcloud.storage.Storage.BucketField; +import com.google.gcloud.storage.StorageException; import com.google.gcloud.storage.testing.RemoteGcsHelper; import org.junit.AfterClass; @@ -41,6 +54,7 @@ import org.junit.Test; import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.InputStream; import java.net.URL; @@ -52,6 +66,7 @@ import java.util.List; import java.util.Map; import java.util.Random; +import java.util.Set; import java.util.concurrent.ExecutionException; import java.util.concurrent.TimeUnit; import java.util.logging.Level; @@ -77,8 +92,9 @@ public static void beforeClass() { @AfterClass public static void afterClass() throws ExecutionException, InterruptedException { - if (storage != null && !RemoteGcsHelper.forceDelete(storage, BUCKET, 5, TimeUnit.SECONDS)) { - if (log.isLoggable(Level.WARNING)) { + if (storage != null) { + boolean wasDeleted = RemoteGcsHelper.forceDelete(storage, BUCKET, 5, TimeUnit.SECONDS); + if (!wasDeleted && log.isLoggable(Level.WARNING)) { log.log(Level.WARNING, "Deletion of bucket {0} timed out, bucket is not empty", BUCKET); } } @@ -86,13 +102,12 @@ public static void afterClass() throws ExecutionException, InterruptedException @Test(timeout = 5000) public void testListBuckets() throws InterruptedException { - Iterator bucketIterator = - storage.list(Storage.BucketListOption.prefix(BUCKET), - Storage.BucketListOption.fields()).values().iterator(); + Iterator bucketIterator = storage.list(Storage.BucketListOption.prefix(BUCKET), + Storage.BucketListOption.fields()).iterateAll(); while (!bucketIterator.hasNext()) { Thread.sleep(500); bucketIterator = storage.list(Storage.BucketListOption.prefix(BUCKET), - Storage.BucketListOption.fields()).values().iterator(); + Storage.BucketListOption.fields()).iterateAll(); } while (bucketIterator.hasNext()) { Bucket remoteBucket = bucketIterator.next(); @@ -273,8 +288,8 @@ public void testGetBlobFailNonExistingGeneration() { assertTrue(remoteBlob.delete()); } - @Test - public void testListBlobsSelectedFields() { + @Test(timeout = 5000) + public void testListBlobsSelectedFields() throws InterruptedException { String[] blobNames = {"test-list-blobs-selected-fields-blob1", "test-list-blobs-selected-fields-blob2"}; ImmutableMap metadata = ImmutableMap.of("k", "v"); @@ -290,14 +305,23 @@ public void testListBlobsSelectedFields() { Blob remoteBlob2 = storage.create(blob2); assertNotNull(remoteBlob1); assertNotNull(remoteBlob2); - Page page = - storage.list(BUCKET, + Page page = storage.list(BUCKET, Storage.BlobListOption.prefix("test-list-blobs-selected-fields-blob"), Storage.BlobListOption.fields(BlobField.METADATA)); - int index = 0; - for (Blob remoteBlob : page.values()) { + // Listing blobs is eventually consistent, we loop until the list is of the expected size. The + // test fails if timeout is reached. + while (Iterators.size(page.iterateAll()) != 2) { + Thread.sleep(500); + page = storage.list(BUCKET, + Storage.BlobListOption.prefix("test-list-blobs-selected-fields-blob"), + Storage.BlobListOption.fields(BlobField.METADATA)); + } + Set blobSet = ImmutableSet.of(blobNames[0], blobNames[1]); + Iterator iterator = page.iterateAll(); + while (iterator.hasNext()) { + Blob remoteBlob = iterator.next(); assertEquals(BUCKET, remoteBlob.bucket()); - assertEquals(blobNames[index++], remoteBlob.name()); + assertTrue(blobSet.contains(remoteBlob.name())); assertEquals(metadata, remoteBlob.metadata()); assertNull(remoteBlob.contentType()); } @@ -305,8 +329,8 @@ public void testListBlobsSelectedFields() { assertTrue(remoteBlob2.delete()); } - @Test - public void testListBlobsEmptySelectedFields() { + @Test(timeout = 5000) + public void testListBlobsEmptySelectedFields() throws InterruptedException { String[] blobNames = {"test-list-blobs-empty-selected-fields-blob1", "test-list-blobs-empty-selected-fields-blob2"}; BlobInfo blob1 = BlobInfo.builder(BUCKET, blobNames[0]) @@ -319,20 +343,120 @@ public void testListBlobsEmptySelectedFields() { Blob remoteBlob2 = storage.create(blob2); assertNotNull(remoteBlob1); assertNotNull(remoteBlob2); - Page page = storage.list( - BUCKET, + Page page = storage.list(BUCKET, Storage.BlobListOption.prefix("test-list-blobs-empty-selected-fields-blob"), Storage.BlobListOption.fields()); - int index = 0; - for (Blob remoteBlob : page.values()) { + // Listing blobs is eventually consistent, we loop until the list is of the expected size. The + // test fails if timeout is reached. + while (Iterators.size(page.iterateAll()) != 2) { + Thread.sleep(500); + page = storage.list(BUCKET, + Storage.BlobListOption.prefix("test-list-blobs-empty-selected-fields-blob"), + Storage.BlobListOption.fields()); + } + Set blobSet = ImmutableSet.of(blobNames[0], blobNames[1]); + Iterator iterator = page.iterateAll(); + while (iterator.hasNext()) { + Blob remoteBlob = iterator.next(); assertEquals(BUCKET, remoteBlob.bucket()); - assertEquals(blobNames[index++], remoteBlob.name()); + assertTrue(blobSet.contains(remoteBlob.name())); assertNull(remoteBlob.contentType()); } assertTrue(remoteBlob1.delete()); assertTrue(remoteBlob2.delete()); } + @Test(timeout = 15000) + public void testListBlobsVersioned() throws ExecutionException, InterruptedException { + String bucketName = RemoteGcsHelper.generateBucketName(); + Bucket bucket = storage.create(BucketInfo.builder(bucketName).versioningEnabled(true).build()); + try { + String[] blobNames = {"test-list-blobs-versioned-blob1", "test-list-blobs-versioned-blob2"}; + BlobInfo blob1 = BlobInfo.builder(bucket, blobNames[0]) + .contentType(CONTENT_TYPE) + .build(); + BlobInfo blob2 = BlobInfo.builder(bucket, blobNames[1]) + .contentType(CONTENT_TYPE) + .build(); + Blob remoteBlob1 = storage.create(blob1); + Blob remoteBlob2 = storage.create(blob2); + Blob remoteBlob3 = storage.create(blob2); + assertNotNull(remoteBlob1); + assertNotNull(remoteBlob2); + assertNotNull(remoteBlob3); + Page page = storage.list(bucketName, + Storage.BlobListOption.prefix("test-list-blobs-versioned-blob"), + Storage.BlobListOption.versions(true)); + // Listing blobs is eventually consistent, we loop until the list is of the expected size. The + // test fails if timeout is reached. + while (Iterators.size(page.iterateAll()) != 3) { + Thread.sleep(500); + page = storage.list(bucketName, + Storage.BlobListOption.prefix("test-list-blobs-versioned-blob"), + Storage.BlobListOption.versions(true)); + } + Set blobSet = ImmutableSet.of(blobNames[0], blobNames[1]); + Iterator iterator = page.iterateAll(); + while (iterator.hasNext()) { + Blob remoteBlob = iterator.next(); + assertEquals(bucketName, remoteBlob.bucket()); + assertTrue(blobSet.contains(remoteBlob.name())); + assertNotNull(remoteBlob.generation()); + } + assertTrue(remoteBlob1.delete()); + assertTrue(remoteBlob2.delete()); + assertTrue(remoteBlob3.delete()); + } finally { + RemoteGcsHelper.forceDelete(storage, bucketName, 5, TimeUnit.SECONDS); + } + } + + @Test(timeout = 5000) + public void testListBlobsCurrentDirectory() throws InterruptedException { + String directoryName = "test-list-blobs-current-directory/"; + String subdirectoryName = "subdirectory/"; + String[] blobNames = {directoryName + subdirectoryName + "blob1", + directoryName + "blob2"}; + BlobInfo blob1 = BlobInfo.builder(BUCKET, blobNames[0]) + .contentType(CONTENT_TYPE) + .build(); + BlobInfo blob2 = BlobInfo.builder(BUCKET, blobNames[1]) + .contentType(CONTENT_TYPE) + .build(); + Blob remoteBlob1 = storage.create(blob1, BLOB_BYTE_CONTENT); + Blob remoteBlob2 = storage.create(blob2, BLOB_BYTE_CONTENT); + assertNotNull(remoteBlob1); + assertNotNull(remoteBlob2); + Page page = storage.list(BUCKET, + Storage.BlobListOption.prefix("test-list-blobs-current-directory/"), + Storage.BlobListOption.currentDirectory()); + // Listing blobs is eventually consistent, we loop until the list is of the expected size. The + // test fails if timeout is reached. + while (Iterators.size(page.iterateAll()) != 2) { + Thread.sleep(500); + page = storage.list(BUCKET, + Storage.BlobListOption.prefix("test-list-blobs-current-directory/"), + Storage.BlobListOption.currentDirectory()); + } + Iterator iterator = page.iterateAll(); + while (iterator.hasNext()) { + Blob remoteBlob = iterator.next(); + assertEquals(BUCKET, remoteBlob.bucket()); + if (remoteBlob.name().equals(blobNames[1])) { + assertEquals(CONTENT_TYPE, remoteBlob.contentType()); + assertEquals(BLOB_BYTE_CONTENT.length, (long) remoteBlob.size()); + assertFalse(remoteBlob.isDirectory()); + } else if (remoteBlob.name().equals(directoryName + subdirectoryName)) { + assertEquals(0L, (long) remoteBlob.size()); + assertTrue(remoteBlob.isDirectory()); + } else { + fail("Unexpected blob with name " + remoteBlob.name()); + } + } + assertTrue(remoteBlob1.delete()); + assertTrue(remoteBlob2.delete()); + } + @Test public void testUpdateBlob() { String blobName = "test-update-blob"; @@ -431,7 +555,7 @@ public void testUpdateBlobFail() { @Test public void testDeleteNonExistingBlob() { String blobName = "test-delete-non-existing-blob"; - assertTrue(!storage.delete(BUCKET, blobName)); + assertFalse(storage.delete(BUCKET, blobName)); } @Test @@ -439,7 +563,7 @@ public void testDeleteBlobNonExistingGeneration() { String blobName = "test-delete-blob-non-existing-generation"; BlobInfo blob = BlobInfo.builder(BUCKET, blobName).build(); assertNotNull(storage.create(blob)); - assertTrue(!storage.delete(BlobId.of(BUCKET, blobName, -1L))); + assertFalse(storage.delete(BlobId.of(BUCKET, blobName, -1L))); } @Test @@ -475,6 +599,37 @@ public void testComposeBlob() { assertNotNull(remoteTargetBlob); assertEquals(targetBlob.name(), remoteTargetBlob.name()); assertEquals(targetBlob.bucket(), remoteTargetBlob.bucket()); + assertNull(remoteTargetBlob.contentType()); + byte[] readBytes = storage.readAllBytes(BUCKET, targetBlobName); + byte[] composedBytes = Arrays.copyOf(BLOB_BYTE_CONTENT, BLOB_BYTE_CONTENT.length * 2); + System.arraycopy(BLOB_BYTE_CONTENT, 0, composedBytes, BLOB_BYTE_CONTENT.length, + BLOB_BYTE_CONTENT.length); + assertArrayEquals(composedBytes, readBytes); + assertTrue(remoteSourceBlob1.delete()); + assertTrue(remoteSourceBlob2.delete()); + assertTrue(remoteTargetBlob.delete()); + } + + @Test + public void testComposeBlobWithContentType() { + String sourceBlobName1 = "test-compose-blob-with-content-type-source-1"; + String sourceBlobName2 = "test-compose-blob-with-content-type-source-2"; + BlobInfo sourceBlob1 = BlobInfo.builder(BUCKET, sourceBlobName1).build(); + BlobInfo sourceBlob2 = BlobInfo.builder(BUCKET, sourceBlobName2).build(); + Blob remoteSourceBlob1 = storage.create(sourceBlob1, BLOB_BYTE_CONTENT); + Blob remoteSourceBlob2 = storage.create(sourceBlob2, BLOB_BYTE_CONTENT); + assertNotNull(remoteSourceBlob1); + assertNotNull(remoteSourceBlob2); + String targetBlobName = "test-compose-blob-with-content-type-target"; + BlobInfo targetBlob = + BlobInfo.builder(BUCKET, targetBlobName).contentType(CONTENT_TYPE).build(); + Storage.ComposeRequest req = + Storage.ComposeRequest.of(ImmutableList.of(sourceBlobName1, sourceBlobName2), targetBlob); + Blob remoteTargetBlob = storage.compose(req); + assertNotNull(remoteTargetBlob); + assertEquals(targetBlob.name(), remoteTargetBlob.name()); + assertEquals(targetBlob.bucket(), remoteTargetBlob.bucket()); + assertEquals(CONTENT_TYPE, remoteTargetBlob.contentType()); byte[] readBytes = storage.readAllBytes(BUCKET, targetBlobName); byte[] composedBytes = Arrays.copyOf(BLOB_BYTE_CONTENT, BLOB_BYTE_CONTENT.length * 2); System.arraycopy(BLOB_BYTE_CONTENT, 0, composedBytes, BLOB_BYTE_CONTENT.length, @@ -558,6 +713,26 @@ public void testCopyBlobUpdateMetadata() { assertTrue(storage.delete(BUCKET, targetBlobName)); } + @Test + public void testCopyBlobNoContentType() { + String sourceBlobName = "test-copy-blob-no-content-type-source"; + BlobId source = BlobId.of(BUCKET, sourceBlobName); + Blob remoteSourceBlob = storage.create(BlobInfo.builder(source).build(), BLOB_BYTE_CONTENT); + assertNotNull(remoteSourceBlob); + String targetBlobName = "test-copy-blob-no-content-type-target"; + ImmutableMap metadata = ImmutableMap.of("k", "v"); + BlobInfo target = BlobInfo.builder(BUCKET, targetBlobName).metadata(metadata).build(); + Storage.CopyRequest req = Storage.CopyRequest.of(source, target); + CopyWriter copyWriter = storage.copy(req); + assertEquals(BUCKET, copyWriter.result().bucket()); + assertEquals(targetBlobName, copyWriter.result().name()); + assertNull(copyWriter.result().contentType()); + assertEquals(metadata, copyWriter.result().metadata()); + assertTrue(copyWriter.isDone()); + assertTrue(remoteSourceBlob.delete()); + assertTrue(storage.delete(BUCKET, targetBlobName)); + } + @Test public void testCopyBlobFail() { String sourceBlobName = "test-copy-blob-source-fail"; @@ -746,6 +921,33 @@ public void testReadAndWriteChannels() throws IOException { assertTrue(storage.delete(BUCKET, blobName)); } + @Test + public void testReadAndWriteChannelsWithDifferentFileSize() throws IOException { + String blobNamePrefix = "test-read-and-write-channels-blob-"; + int[] blobSizes = {0, 700, 1024 * 256, 2 * 1024 * 1024, 4 * 1024 * 1024, 4 * 1024 * 1024 + 1}; + Random rnd = new Random(); + for (int blobSize : blobSizes) { + String blobName = blobNamePrefix + blobSize; + BlobInfo blob = BlobInfo.builder(BUCKET, blobName).build(); + byte[] bytes = new byte[blobSize]; + rnd.nextBytes(bytes); + try (WriteChannel writer = storage.writer(blob)) { + writer.write(ByteBuffer.wrap(bytes)); + } + ByteArrayOutputStream output = new ByteArrayOutputStream(); + try (ReadChannel reader = storage.reader(blob.blobId())) { + ByteBuffer buffer = ByteBuffer.allocate(64 * 1024); + while (reader.read(buffer) > 0) { + buffer.flip(); + output.write(buffer.array(), 0, buffer.limit()); + buffer.clear(); + } + } + assertArrayEquals(bytes, output.toByteArray()); + assertTrue(storage.delete(BUCKET, blobName)); + } + } + @Test public void testReadAndWriteCaptureChannels() throws IOException { String blobName = "test-read-and-write-capture-channels-blob"; @@ -955,7 +1157,7 @@ public void testDeleteBlobsFail() { assertNotNull(storage.create(sourceBlob1)); List deleteStatus = storage.delete(sourceBlob1.blobId(), sourceBlob2.blobId()); assertTrue(deleteStatus.get(0)); - assertTrue(!deleteStatus.get(1)); + assertFalse(deleteStatus.get(1)); } @Test diff --git a/gcloud-java/README.md b/gcloud-java/README.md index 4a581d56f991..e296d0c0c565 100644 --- a/gcloud-java/README.md +++ b/gcloud-java/README.md @@ -7,6 +7,7 @@ Java idiomatic client for [Google Cloud Platform][cloud-platform] services. [![Coverage Status](https://coveralls.io/repos/GoogleCloudPlatform/gcloud-java/badge.svg?branch=master)](https://coveralls.io/r/GoogleCloudPlatform/gcloud-java?branch=master) [![Maven](https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java.svg)]( https://img.shields.io/maven-central/v/com.google.gcloud/gcloud-java.svg) [![Codacy Badge](https://api.codacy.com/project/badge/grade/9da006ad7c3a4fe1abd142e77c003917)](https://www.codacy.com/app/mziccard/gcloud-java) +[![Dependency Status](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969/badge.svg?style=flat)](https://www.versioneye.com/user/projects/56bd8ee72a29ed002d2b0969) - [Homepage] (https://googlecloudplatform.github.io/gcloud-java/) - [API Documentation] (http://googlecloudplatform.github.io/gcloud-java/apidocs) @@ -26,16 +27,16 @@ If you are using Maven, add this to your pom.xml file com.google.gcloud gcloud-java - 0.1.3 + 0.1.5 ``` If you are using Gradle, add this to your dependencies ```Groovy -compile 'com.google.gcloud:gcloud-java:0.1.3' +compile 'com.google.gcloud:gcloud-java:0.1.5' ``` If you are using SBT, add this to your dependencies ```Scala -libraryDependencies += "com.google.gcloud" % "gcloud-java" % "0.1.3" +libraryDependencies += "com.google.gcloud" % "gcloud-java" % "0.1.5" ``` Troubleshooting diff --git a/gcloud-java/pom.xml b/gcloud-java/pom.xml index 4d46ff7fd0f0..654b34f92056 100644 --- a/gcloud-java/pom.xml +++ b/gcloud-java/pom.xml @@ -10,7 +10,7 @@ com.google.gcloud gcloud-java-pom - 0.1.4-SNAPSHOT + 0.1.6-SNAPSHOT @@ -28,6 +28,11 @@ gcloud-java-datastore ${project.version} + + ${project.groupId} + gcloud-java-dns + ${project.version} + ${project.groupId} gcloud-java-resourcemanager diff --git a/pom.xml b/pom.xml index 3cfa57b45c96..f1f4fc4668d7 100644 --- a/pom.xml +++ b/pom.xml @@ -4,7 +4,7 @@ com.google.gcloud gcloud-java-pom pom - 0.1.4-SNAPSHOT + 0.1.6-SNAPSHOT GCloud Java https://github.com/GoogleCloudPlatform/gcloud-java @@ -98,6 +98,7 @@ gcloud-java-contrib gcloud-java-core gcloud-java-datastore + gcloud-java-dns gcloud-java-examples gcloud-java-pubsub gcloud-java-resourcemanager @@ -134,7 +135,7 @@ org.apache.maven.plugins maven-surefire-plugin - 2.18 + 2.19.1 @@ -216,6 +217,13 @@ + + + + test-jar + + + maven-compiler-plugin @@ -233,7 +241,7 @@ org.apache.maven.plugins maven-source-plugin - 2.4 + 3.0.0 attach-sources @@ -273,7 +281,7 @@ org.sonatype.plugins nexus-staging-maven-plugin - 1.6.5 + 1.6.6 true sonatype-nexus-staging @@ -284,7 +292,7 @@ org.eluder.coveralls coveralls-maven-plugin - 3.1.0 + 4.1.0 ${basedir}/target/coverage.xml @@ -393,6 +401,24 @@ protected true ${project.build.directory}/javadoc + + + API packages + com.google.gcloud* + + + Test helpers packages + com.google.gcloud.bigquery.testing:com.google.gcloud.datastore.testing:com.google.gcloud.resourcemanager.testing:com.google.gcloud.storage.testing + + + Example packages + com.google.gcloud.examples* + + + SPI packages + com.google.gcloud.spi:com.google.gcloud.bigquery.spi:com.google.gcloud.datastore.spi:com.google.gcloud.resourcemanager.spi:com.google.gcloud.storage.spi + + diff --git a/src/site/site.xml b/src/site/site.xml index 55047ce85c54..6279179eb389 100644 --- a/src/site/site.xml +++ b/src/site/site.xml @@ -20,7 +20,7 @@ org.apache.maven.skins maven-fluido-skin - 1.3.1 + 1.4